You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/08 00:49:28 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8220
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8220/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12838] Update artifact local path for DataflowRunner Java
[Valentyn Tymofieiev] Disable Kafka perf tests.
[heejong] separate into resolveArtifacts method
[heejong] add test
[Andrew Pilloud] [BEAM-12850] Calcite drops empty Calc now
[Andrew Pilloud] [BEAM-12853] VALUES produces a UNION, window can't be set afterwards
[Andrew Pilloud] [BEAM-12852] Revert BigTable changes, just cast to bigint
[heejong] update
[heejong] fix formatting
[Andrew Pilloud] [BEAM-12851] Map output table names
[Luke Cwik] [BEAM-12802] Define a prefetchable iterator and iterable and utility
------------------------------------------
[...truncated 104.00 KB...]
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:01f80266ae7a5a0f03d1f10883467330c2042fe937251e40f346e49651922a95
Associated tags:
- 20210908-001215233740731
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210908-001215233740731
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210908-001215233740731].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:01f80266ae7a5a0f03d1f10883467330c2042fe937251e40f346e49651922a95].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:8d74b22db279c0b3710e797836f4d28ba2982a6d09cc47eb8ab8b4567c95725d
Associated tags:
- b3929772-af5a-41b8-bb22-4d64cc141918
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:b3929772-af5a-41b8-bb22-4d64cc141918].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:8d74b22db279c0b3710e797836f4d28ba2982a6d09cc47eb8ab8b4567c95725d].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:286a4e7bb8ef5208ea795e3ed5e3af93b1e82c7bfcea2e78c86ba3fa2de9bd91
Associated tags:
- 1cb05c49-a897-475c-a80d-f371f55fadad
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:1cb05c49-a897-475c-a80d-f371f55fadad].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:286a4e7bb8ef5208ea795e3ed5e3af93b1e82c7bfcea2e78c86ba3fa2de9bd91].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:541: in run
self._options).run(False)
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.test_dataflow_runner.TestDataflowRunner object at 0x7eff691b99b0>
pipeline = <apache_beam.pipeline.Pipeline object at 0x7eff68884630>
options = <apache_beam.options.pipeline_options.PipelineOptions object at 0x7eff6883bdd8>
def run_pipeline(self, pipeline, options):
"""Execute test pipeline and verify test matcher"""
test_options = options.view_as(TestOptions)
on_success_matcher = test_options.on_success_matcher
wait_duration = test_options.wait_until_finish_duration
is_streaming = options.view_as(StandardOptions).streaming
# [BEAM-1889] Do not send this to remote workers also, there is no need to
# send this option to remote executors.
test_options.on_success_matcher = None
self.result = super(TestDataflowRunner,
self).run_pipeline(pipeline, options)
if self.result.has_job:
# TODO(markflyhigh)(BEAM-1890): Use print since Nose dosen't show logs
# in some cases.
print('Worker logs: %s' % self.build_console_url(options))
try:
self.wait_until_in_state(PipelineState.RUNNING)
if is_streaming and not wait_duration:
_LOGGER.warning('Waiting indefinitely for streaming job.')
self.result.wait_until_finish(duration=wait_duration)
if on_success_matcher:
from hamcrest import assert_that as hc_assert_that
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
E AssertionError:
E Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
E but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: FAILED
apache_beam/runners/dataflow/test_dataflow_runner.py:69: AssertionError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> -
============== 1 failed, 3 passed, 2 warnings in 1695.78 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210908-001219271197573
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b12a82615790729511c606fb25fe7df5755087a02b7a9e1f2fdad66e17bb0671
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:a1b330b4-e3c3-4df4-aab7-e84082267d51
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a46fd032602e3b9988499bee8c499a981ee204bc1784edc400fdec7dd175778c
Deleted: sha256:224ccfac6de64b806272945e3013d9bf3887f60bdb3ccdb974b6ea5800624dff
Deleted: sha256:5afeeeb87feecde06674ac7a1e14a723206bfdf1134763331cf1ea5c3d83edae
Deleted: sha256:7f11a051f377111f215cb467e859c21b5874e0e631a83776ce84e8b3c2ba8f24
Deleted: sha256:017e0b020e2de719cb23a7a226ab36a08df0c566421c61c232baeca7a1827e80
Deleted: sha256:156d80e8713c37b89f02b9d774a0c0d8c4152a95f5c1f981f79e90651fdf9d98
Deleted: sha256:f518af2b06c9b6ddac9107ad5cf895afea3cbd6058342c241958fabb6b027adc
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b12a82615790729511c606fb25fe7df5755087a02b7a9e1f2fdad66e17bb0671
Associated tags:
- 20210908-001219271197573
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210908-001219271197573
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210908-001219271197573].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b12a82615790729511c606fb25fe7df5755087a02b7a9e1f2fdad66e17bb0671].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a46fd032602e3b9988499bee8c499a981ee204bc1784edc400fdec7dd175778c
Associated tags:
- a1b330b4-e3c3-4df4-aab7-e84082267d51
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:a1b330b4-e3c3-4df4-aab7-e84082267d51].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a46fd032602e3b9988499bee8c499a981ee204bc1784edc400fdec7dd175778c].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:62ce37b993e913e67c28c147d76b56e5836732557144618ff22c7085aef0afa6
Associated tags:
- 09b9db5a-6166-4be1-809f-15b2c4e215ff
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:09b9db5a-6166-4be1-809f-15b2c4e215ff].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:62ce37b993e913e67c28c147d76b56e5836732557144618ff22c7085aef0afa6].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:a1cd31be2777997f791101e96b16d687adc05328ec8eeca2e8f9c51dcd71e191
Associated tags:
- 20210908-001211511303459
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210908-001211511303459
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210908-001211511303459].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:a1cd31be2777997f791101e96b16d687adc05328ec8eeca2e8f9c51dcd71e191].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:0585aa75edff7d5e5199fbb6e62018cd4ba73f9b0cc48bf41911500a07ce7c73
Associated tags:
- aaeca33e-400b-4ba5-9d11-02dcf8451791
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:aaeca33e-400b-4ba5-9d11-02dcf8451791].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:0585aa75edff7d5e5199fbb6e62018cd4ba73f9b0cc48bf41911500a07ce7c73].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:a615a7e36a40baf32bc0803c935c6f44c8cb2c4856c6bc1e9357b692ad9d0760
Associated tags:
- 2a13fad7-75e6-41fd-9b6c-1e1826a5ecd9
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:2a13fad7-75e6-41fd-9b6c-1e1826a5ecd9].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:a615a7e36a40baf32bc0803c935c6f44c8cb2c4856c6bc1e9357b692ad9d0760].
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 48m 56s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/tiuhztnvpjjlm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #8221
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8221/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org