You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/01 19:27:06 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8195
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8195/display/redirect?page=changes>
Changes:
[baeminbo] Fix apiclient_test unittest not to fail with no credentials
[noreply] [BEAM-10913] - Adding new Grafana dashboard to monitor GAs post-commit
------------------------------------------
[...truncated 102.36 KB...]
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:43479055d1c75446ea0d065447e2d3638de2b8bb991626bf37fcfe567196d601
Associated tags:
- 20210901-181510197198587
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210901-181510197198587
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210901-181510197198587].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:43479055d1c75446ea0d065447e2d3638de2b8bb991626bf37fcfe567196d601].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2626b41b04760f0aa85e2eb7685af48c5fb8fa5c43f19708af4b65e7542c6837
Associated tags:
- 7a18632a-25cb-4f8b-bb16-16a07ebabc18
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:7a18632a-25cb-4f8b-bb16-16a07ebabc18].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2626b41b04760f0aa85e2eb7685af48c5fb8fa5c43f19708af4b65e7542c6837].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:957ad836324bda5b2bb27bc4c23058a1032f4423a7074f44c25450d397b73cdd
Associated tags:
- 8967be7e-6a22-4961-bfae-2703862c35c5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:8967be7e-6a22-4961-bfae-2703862c35c5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:957ad836324bda5b2bb27bc4c23058a1032f4423a7074f44c25450d397b73cdd].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
[gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.7.3 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:110: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:586: in __exit__
self.result = self.run()
apache_beam/pipeline.py:541: in run
self._options).run(False)
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210901183315784926-9813'
createTime: '2021-09-01T18:33:17.687638Z'
...021-09-01T18:33:17.687638Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7efd2852add8>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
> time.sleep(5.0)
E Failed: Timeout >900.0s
apache_beam/runners/dataflow/dataflow_runner.py:1621: Failed
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> -
============== 1 failed, 3 passed, 2 warnings in 3178.90 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210901-181501581238085
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:81ed38b5a9900cc4a13bdffd8b8abe21a548f3cb31869a0dfe99da73684dfc30
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:9d2f8b70-a1bd-44d9-8e93-5bee26e5eb08
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:4513661eb3a563cf7965dc4da7132767db4672391afd3d69a4f2e78b1e03e362
Deleted: sha256:01c79062e3c0df047786eb1e21d7d16e085299f495e474a7439e58327f4e2231
Deleted: sha256:90e5681a30016bee6d7a124c1ffa7f1f78b38ed9aad7b38a6ca1a71d68220180
Deleted: sha256:3506a1418efa34bece818a92f208fb0a80cb6d27d21d19e4fba9c84ed0a7ecb9
Deleted: sha256:be4376fddf83c5fa109651527346637ff05bc69c01ea54aa4c4f3048d04ee413
Deleted: sha256:6c3797bedee3e109c154af35d417c1bf932c4761ddc481a0f56737d6c999e9ce
Deleted: sha256:4c6617ebb753f05d6300629266a9a45c817d80a6b5bc0163ebe398acec1829ad
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:81ed38b5a9900cc4a13bdffd8b8abe21a548f3cb31869a0dfe99da73684dfc30
Associated tags:
- 20210901-181501581238085
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210901-181501581238085
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210901-181501581238085].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:81ed38b5a9900cc4a13bdffd8b8abe21a548f3cb31869a0dfe99da73684dfc30].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:4513661eb3a563cf7965dc4da7132767db4672391afd3d69a4f2e78b1e03e362
Associated tags:
- 9d2f8b70-a1bd-44d9-8e93-5bee26e5eb08
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:9d2f8b70-a1bd-44d9-8e93-5bee26e5eb08].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:4513661eb3a563cf7965dc4da7132767db4672391afd3d69a4f2e78b1e03e362].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:8ef5aa21d64f5c258dcb56fe7a9c33e535ce6ab1b0bb45d906bf88f80e0b8acd
Associated tags:
- 71904cbd-df45-4b5c-94af-76a551ffe505
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:71904cbd-df45-4b5c-94af-76a551ffe505].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:8ef5aa21d64f5c258dcb56fe7a9c33e535ce6ab1b0bb45d906bf88f80e0b8acd].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b5161823730b228323c5cb368446d1c133ccdb43ac161cf7613dc49e38c36706
Associated tags:
- 20210901-181510330017159
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210901-181510330017159
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210901-181510330017159].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b5161823730b228323c5cb368446d1c133ccdb43ac161cf7613dc49e38c36706].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:c258638f760f74acaabda003c962725f85edde697052af12c132ea94a8a86a1a
Associated tags:
- 8dba070b-b9c8-407e-b2a4-81c1c32f4115
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:8dba070b-b9c8-407e-b2a4-81c1c32f4115].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:c258638f760f74acaabda003c962725f85edde697052af12c132ea94a8a86a1a].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:12930e161605fd01d67acb1a414e15e570e2a8974ba347e65b50fab8201900b3
Associated tags:
- 0e2a7074-fb6e-4aee-82a9-06bd3c65c8cb
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:0e2a7074-fb6e-4aee-82a9-06bd3c65c8cb].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:12930e161605fd01d67acb1a414e15e570e2a8974ba347e65b50fab8201900b3].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 26m 36s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/h2b6q5vzs243o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #8196
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8196/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org