You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/10/29 00:54:31 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8424
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8424/display/redirect?page=changes>
Changes:
[Brian Hulette] [BEAM-13099] Replace call to RelNode.metadata with BeamRelMetadataQuery
[Brian Hulette] [BEAM-13099] Update all rels to use BeamRelMetadataQuery
------------------------------------------
[...truncated 116.96 KB...]
Deleted: sha256:c2c3a0d42b52c40373d70564c71f3c1ab607a16002e013c7f005b3375ca59e26
Deleted: sha256:e48147cb9f43d1f2f14a70c6743c092c9fd786063d5ca98dabdd255716f9a9be
Deleted: sha256:51ae2bcf371b8301e93075fafb07a4bd62c9fc3e2e1b1ffb206aada2b3f1d6d0
Deleted: sha256:4cddd1c732825e5143b01341c0dd4a3ff1dea03d2beb22c8222698bb67c9d74c
Deleted: sha256:697f0e16e134a27eec13cc8d15a506743cdef38425faa7b2fc6f0bcebdcc0109
Deleted: sha256:32ba27c96c22a909be6fc0eb23b876b70b61ea8d3f4cc7e9b7ab14575545947a
Deleted: sha256:c3db38635970036c3cc2c2e9ce6e1158bb004b384907910b5f9bb1e050f3c90b
Deleted: sha256:4b9a6fc9b059c2e2d2e107e38be9389b81eea78d3c234a8ece31612600694bab
Deleted: sha256:49f47637ce603e4e0627be0341e47873fed3a5309e38da267786a571590cc265
Deleted: sha256:da0ba954b11984844cb58b799d0491b315a2c9689e69b42cdbffee50fa38317e
Deleted: sha256:da9f278952b6fabbf3efaa386ec58630e9fcae22a2a04aaf391a38da1bc35aee
Deleted: sha256:445cecefbbc11bb6ca161d32fd2f54c547665b6a3a0529c526008d5b550c3205
Deleted: sha256:e3ca8bf4c44f4c366e36ae6f2e594774a7863ba97b96d6b4c5051c1644ad69b7
Deleted: sha256:060a890b450b00ae86d81cfad14ed70da6edf55ed9d9b0500ca7e4752b03fc81
Deleted: sha256:4d6b497b4f816defbca85050d7dc0822993eb3164cc160f4c28c058125d2caef
Deleted: sha256:4581d6e63bde791b88da7da81aaee97a79f916532c12673e0df0a7bda197d3ee
Deleted: sha256:3c1e1d87b40f355fc8cebc113ea54be8779586f78e2bf35b3a7c16a36e36e6a7
Deleted: sha256:6cd223ba2133a92e16c655720985aa5a4757fd6ba9ce5623f3c46d6407bc30cf
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:3a5207385f0d4ffed2ed53e26dd759b837311817b61090529829bd9b7232370b
Associated tags:
- 20211029-001532585200026
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211029-001532585200026
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211029-001532585200026].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:3a5207385f0d4ffed2ed53e26dd759b837311817b61090529829bd9b7232370b].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:9d093d16af8d641535e2d15598379628b4424c05dce674b0c36c1ccc39278bca
Associated tags:
- 9a6aaf7e-e497-4bad-b7ff-ff13285d5b20
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:9a6aaf7e-e497-4bad-b7ff-ff13285d5b20].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:9d093d16af8d641535e2d15598379628b4424c05dce674b0c36c1ccc39278bca].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a420c448e2a5cd8dd402ef6879e1303f27963e953a8e18549f3a90d6f0ab4957
Associated tags:
- b864d317-4a32-4eec-8004-c4ce355fbd7a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:b864d317-4a32-4eec-8004-c4ce355fbd7a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:a420c448e2a5cd8dd402ef6879e1303f27963e953a8e18549f3a90d6f0ab4957].
Removed the container
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:f254f8769190774569756b56eac1360e52141b073221612ad2afe18ab82cb174
Associated tags:
- 20211029-001539905788928
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211029-001539905788928
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211029-001539905788928].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:f254f8769190774569756b56eac1360e52141b073221612ad2afe18ab82cb174].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:634c3f4614c89103cc7fda4b2af8bb9c3a38b25c289e944c06b5d41a48702673
Associated tags:
- 4887789e-6eaa-4b4a-8901-725c74c09b82
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:4887789e-6eaa-4b4a-8901-725c74c09b82].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:634c3f4614c89103cc7fda4b2af8bb9c3a38b25c289e944c06b5d41a48702673].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2325bdbb772661392720b15b88add9ea6fe5b0ac3c21a7bcc3c2941a984b98b
Associated tags:
- 81083caa-6234-47cd-84e2-1ca4f303f4ac
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:81083caa-6234-47cd-84e2-1ca4f303f4ac].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2325bdbb772661392720b15b88add9ea6fe5b0ac3c21a7bcc3c2941a984b98b].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
[gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.7.10 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:109: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:549: in run
self._options).run(False)
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:64: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20211029003348467103-9813'
createTime: '2021-10-29T00:33:50.078224Z'
...021-10-29T00:33:50.078224Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7fe517823f50>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
> time.sleep(5.0)
E Failed: Timeout >900.0s
apache_beam/runners/dataflow/dataflow_runner.py:1629: Failed
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml> -
============== 1 failed, 3 passed, 2 warnings in 2220.14 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211029-001529113787414
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:f7ca106de930aaaf9d1a55eef6f89c74bf77b2e45784aada5700a822389456b8
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:c6720fa7-90e2-49e8-b169-54e60063676a
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b2fbfa65083fb6329bbadbf411ff78a2c4cab0d91b57df57f8eeb2f8bb9f28e5
Deleted: sha256:9044e7095601666ebe1b46f63c0779c3c34fe15e9939f29ddca3236936948a07
Deleted: sha256:b7ad3168231484da3811edd364452dc76b325f8fd3baee1c7512262a885f5fe1
Deleted: sha256:d96e8e795b8763d264ebdc7355dda2aa5f1f942c6a2a8bc2b141c62ffac900f6
Deleted: sha256:744b77b8424cbe461358a5b6fe8d3a7935a7dafb3fa607146999fc57c7646753
Deleted: sha256:9c37bf2defaefb3fbf953a7bf8ed9a0294eef4eca149c370368e5806355deed3
Deleted: sha256:5bd798eedd75095e196ec1211f43d3a643e6d6527a89e4203b3fb1d9faab9837
Deleted: sha256:d6c09fc5854a03fb33d5cd6541dff653672689120809f8f5260e9f6d7dd2b24e
Deleted: sha256:c4b9829facadd227dd1b2e164f475c5f6f577afaf3f856bdf72e24d61f64b663
Deleted: sha256:aa11712d9faae6b04ef4a533a22f570a092ee68a8c4fd6212e5e7882029a76e9
Deleted: sha256:546a6651af7d92dc27230e82c3e4d9b898f58f91143712a4b64996444919e9ef
Deleted: sha256:ca7760be84bcd82cce51463325f0daf48e7d7b07db1bcbd7fbafd86e946caded
Deleted: sha256:542393e8da68fa1578874450315921767df9d5c3ff50ab3572c5736f1473a391
Deleted: sha256:639652185447da9d5ad3724d7490d694a07dbd3f8a4888ebb4f0c4e2b84523a1
Deleted: sha256:42cb987ff2fa7c1f681175f868e0bf6cf7c3eba15aeaec60101a0df459abd616
Deleted: sha256:938a452404b433dfbf9e822cf9957d75db708eee4bb1a09b9fa04089a02df518
Deleted: sha256:5ffbcceec81a3a20fddf7645f722865243f6418a098804eeff3f3d5d461a53bf
Deleted: sha256:f55ebd8eeff9dec8af79672dc9e7efaa2e4e1efa0eaaa5949024372521ab3b8c
Deleted: sha256:439779ab942197f0bfe5d4b514fcfe7d5a0459d351867bc4859bca9d64b8e157
Deleted: sha256:3e37499dc4367d63054eea5286b64c5d05a4eee845daddea994bbedf7518b24a
Deleted: sha256:76e60562cff7ab02d8131a12fc7efe841f921e9fb67b13f186ccb506b48606ac
Deleted: sha256:80462bf980298a155feb573f838ae9ee2c94eb200dc980a4cb7d9de2a102a0a9
Deleted: sha256:93f9116ade9b6e3d625ab3d6b968e3102957f8e12a9deb4229637aaf39b2dcf7
Deleted: sha256:30c1f3ab5e979058bd32910309b1d334dd8590d50f60270c171b3aa151368c78
Deleted: sha256:a499f3f844b156a3eace6ae5f9593b4e5943432b59611de2fe3cbdf8397c0705
Deleted: sha256:0aba31bbbc3ca1583a06d238640489ea0373bf67a8f22cf2d7788391d942acf9
Deleted: sha256:caa25c06e6fe6cf6315a5f9cdc3e2a1c9990e80bf23cc96795fbe58af2d9e3bc
Deleted: sha256:0d9f4acb82610f1c9076f5a2bf18a0f8fb12b2373d5ffecc9c2acfae96c87419
Deleted: sha256:a8c3f2cc146c7b7191eacff05b730dc0a339ca9ee16b34d886bd5ed4cc0f48f2
Deleted: sha256:e73cd65d25bc31d0cbef5bee85a0f7140120f1da2d81295e0efd189bcd68ecd1
Deleted: sha256:68a28083b3abba0679cecee6799f1f2a39f7378d2ca3da4fad5cfbbc916923ed
Deleted: sha256:a640d7b662f667fc4fef33bf2129557354fa4652578b5c4df90c5a40bc74bb02
Deleted: sha256:9c23c373c3b26e816a81bb5f7dc8e52bcaca9d439aa91b6f4f7fc56b0c570c6f
Deleted: sha256:b78980d358b198e727ed8be45c326edce4f27bb8ff8e6b50b59769863db79e3c
Deleted: sha256:5b309af4cc3d8299a7d08e6a6f62cb0a94c5c776a0ff783c2cb3b0a1bdf31c9e
Deleted: sha256:326209bc4f794d58877a9bc4edb632dfbbdacc8e35d5f7e186beb30f9859422e
Deleted: sha256:4238f3d663740871631da2abec421916ade46ccd6ebea06362ce1142b9270c70
Deleted: sha256:7dab4e5aa50aa22f07131ae40e39de7ebb6f3dca84e755b14b9c57c37e0266e8
Deleted: sha256:0b5e9a6e2ecc206356458cf301ac248143a6d91881237220f038427ba06b7bd5
Deleted: sha256:52ee1b36377dc978ff99de05f1d89eb5364b61193c805fdad9ae720f441c7cdf
Deleted: sha256:7d64cdadc9c325943b12a94c6508557df18546e437379e8a61a4a61b303b991a
Deleted: sha256:54f25cafbc46381a375349c84d7cf015801e25bd7a7d17fa18e466c94c2f622d
Deleted: sha256:9c377b129eadabbdbf82c608757c3ca4bb2ee963d558dd2b4784facec7ca1133
Deleted: sha256:450cc3ada450ace056d3010da4b372c754e0ac996bd9f289c69728a695be3720
Deleted: sha256:74a4e17f8e4bdac5719eaf0daf28b84d2ab04d703de0f90acfcfe687956cb34b
Deleted: sha256:fce41be0cc2ebae73f7e6d82ca23ba885a1a4a7edefbd504349f77b7b55a9fd4
Deleted: sha256:b83bc04513a2cfc093770647c22b731d3bc31c93883b57cf8a234b087aa7f12b
Deleted: sha256:aff37b96898d5242180187f48040943a434c6b7b9faab94dabb04267d958394b
Deleted: sha256:5656664ffe2bdf57e54b03a8600873469934282005ec82284885ea4a2873c8cc
Deleted: sha256:9faaf29a6c63e58728861e68357e43be1ad33723f9fe00c9091b0f8654ae4d46
Deleted: sha256:5deeedb0d01a55b45df566dca1edead055e0c1b3a4485095b7fb257cb7c9f6af
Deleted: sha256:f2b30e1519fee7abff3719abd61ef9b36610c9b9ea73d088be0de316237f63fd
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:f7ca106de930aaaf9d1a55eef6f89c74bf77b2e45784aada5700a822389456b8
Associated tags:
- 20211029-001529113787414
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211029-001529113787414
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211029-001529113787414].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:f7ca106de930aaaf9d1a55eef6f89c74bf77b2e45784aada5700a822389456b8].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b2fbfa65083fb6329bbadbf411ff78a2c4cab0d91b57df57f8eeb2f8bb9f28e5
Associated tags:
- c6720fa7-90e2-49e8-b169-54e60063676a
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:c6720fa7-90e2-49e8-b169-54e60063676a].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:b2fbfa65083fb6329bbadbf411ff78a2c4cab0d91b57df57f8eeb2f8bb9f28e5].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:f5f2aed2953edc8def257b9d466f72edf631a40ff857a0d79d535cffb58e6838
Associated tags:
- 98a32f82-3755-4c12-b518-bf19fa939114
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:98a32f82-3755-4c12-b518-bf19fa939114].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:f5f2aed2953edc8def257b9d466f72edf631a40ff857a0d79d535cffb58e6838].
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 54m 2s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/g7e5hbel3bsdg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #8426
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8426/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #8425
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8425/display/redirect?page=changes>
Changes:
[artur.khanin] Implemented GetCompileOutput and covered with unit tests
[artur.khanin] Replaced logging library regarding PR comment
[aydar.zaynutdinov] [BEAM-13120][Playground]
[Kyle Weaver] [BEAM-13143] Fix python doc generator error.
[Pablo Estrada] Handle runner-provided shards for TextIO
[noreply] Merge pull request #15721 from [BEAM-13023][Playground] Implement Redis
[noreply] Merge pull request #15802 from [BEAM-12970][Playground] Implement gRPC
[noreply] Merge pull request #15800 from [BEAM-12970][Playground] Implement gRPC
------------------------------------------
[...truncated 98.87 KB...]
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
function cleanup_container {
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG || echo "Failed to remove container image"
for image in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH)
do docker rmi $image || echo "Failed to remove prebuilt sdk container image"
done
gcloud --quiet container images delete $CONTAINER:$TAG || echo "Failed to delete container"
for digest in $(gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)")
do gcloud container images delete $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk@$digest --force-delete-tags --quiet || echo "Failed to remove prebuilt sdk container image"
done
echo "Removed the container"
}
trap cleanup_container EXIT
echo ">>> Successfully built and push container $CONTAINER"
cd sdks/python
SDK_LOCATION=$2
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
pytest -o junit_suite_name=$IMAGE_NAME \
-m="it_validatescontainer" \
--show-capture=no \
--numprocesses=1 \
--timeout=900 \
--junitxml=$XUNIT_FILE \
--ignore-glob '.*py3\d?\.py$' \
--log-cli-level=INFO \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--sdk_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=$PREBUILD_SDK_CONTAINER_REGISTRY_PATH"
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:09c241bec4d88d78d92d837ada16679b593cc9c49149d2746b1f6600edc722f3
Associated tags:
- 20211029-061340533775248
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211029-061340533775248
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211029-061340533775248].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:09c241bec4d88d78d92d837ada16679b593cc9c49149d2746b1f6600edc722f3].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:7be2f8746e043a68b2ce1382b2b74a059505af6a717642341e83a328446c5c58
Associated tags:
- 59a3cd7b-688a-4d8a-b4b2-dfe720123c2b
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:59a3cd7b-688a-4d8a-b4b2-dfe720123c2b].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:7be2f8746e043a68b2ce1382b2b74a059505af6a717642341e83a328446c5c58].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:36562a0e9e320fda38c5b2336d26e65b6ba60aa5620a5715e2c65a1cb85c0616
Associated tags:
- d982fc73-aedd-45dd-8032-2b78cb785100
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:d982fc73-aedd-45dd-8032-2b78cb785100].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:36562a0e9e320fda38c5b2336d26e65b6ba60aa5620a5715e2c65a1cb85c0616].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST"
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:731cf26855353b0335dc6c1f4670815eda8c259af4d920864f88c17bcc626e0f
Associated tags:
- 20211029-061340569479127
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211029-061340569479127
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211029-061340569479127].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:731cf26855353b0335dc6c1f4670815eda8c259af4d920864f88c17bcc626e0f].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:fbfde8360e0e4d5dd464bf3efb4ae7bd46e339b1944c91a4f43e573ff5f4760e
Associated tags:
- 1cc79a46-557a-4fbd-8441-473b6608d6c0
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:1cc79a46-557a-4fbd-8441-473b6608d6c0].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:fbfde8360e0e4d5dd464bf3efb4ae7bd46e339b1944c91a4f43e573ff5f4760e].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:69dd1331abafb2e54f85299c1983b64cee35b28feac38f6725a81791f3ed6fee
Associated tags:
- 1a35ccc0-9a0b-4ffb-9a46-b704f22d16f5
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:1a35ccc0-9a0b-4ffb-9a46-b704f22d16f5].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:69dd1331abafb2e54f85299c1983b64cee35b28feac38f6725a81791f3ed6fee].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
[gw0] [100%] PASSED apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self = <apache_beam.examples.wordcount_it_test.WordCountIT testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:109: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:549: in run
self._options).run(False)
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:64: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20211029063226682438-9813'
createTime: '2021-10-29T06:32:28.474091Z'
...021-10-29T06:32:28.474091Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7fbcba773160>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
> time.sleep(5.0)
E Failed: Timeout >900.0s
apache_beam/runners/dataflow/dataflow_runner.py:1629: Failed
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml> -
============== 1 failed, 3 passed, 2 warnings in 2270.63 seconds ===============
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211029-061340554899825
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:51b4bc4c3e9806a3953d0bca36d6168ef8b512c2fd99f6f60cc64686fe67dece
docker images --format '{{.Repository}}:{{.Tag}}' | grep $PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:4774acb9-13a5-486d-ad8d-c6934d1ae219
Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:2da9627881f19b180f4e000ba038785faa97b06aac259c31f039256cc1ebc11a
Deleted: sha256:f1f5785ab2bdb21affcd582c124020c91ec3bf498dbc4d02e23bc2991788509a
Deleted: sha256:b6c7a5bcc1eadb35421dc9972a26b467acbd7dc719b299a66a0293631181a68c
Deleted: sha256:06e8f0006c27d44429fec6444c30e91522ce007a5ba9718dbc1e99dc9d247c74
Deleted: sha256:9bda8480ef914c8c71d0abd5de1a6f80a1af61829f3a9d482d3b8b61f780555e
Deleted: sha256:07e2823336ff842dd6c3e42a3c828543ecd6762bd40d915b7352f90b20fb5994
Deleted: sha256:1c9cab04be130a454cd15d7168d6e669d7b659a886141e6394667f6591ed3e35
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:51b4bc4c3e9806a3953d0bca36d6168ef8b512c2fd99f6f60cc64686fe67dece
Associated tags:
- 20211029-061340554899825
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211029-061340554899825
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211029-061340554899825].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:51b4bc4c3e9806a3953d0bca36d6168ef8b512c2fd99f6f60cc64686fe67dece].
gcloud container images list-tags $PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk --format="get(digest)"
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:2da9627881f19b180f4e000ba038785faa97b06aac259c31f039256cc1ebc11a
Associated tags:
- 4774acb9-13a5-486d-ad8d-c6934d1ae219
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:4774acb9-13a5-486d-ad8d-c6934d1ae219].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:2da9627881f19b180f4e000ba038785faa97b06aac259c31f039256cc1ebc11a].
Digests:
- us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:1776307dbad63ff44654af01686051b6f997409fded66b6c4422dce43fe23d25
Associated tags:
- b2b4b380-bd2c-4eb4-a400-0b18cc57ea11
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:b2b4b380-bd2c-4eb4-a400-0b18cc57ea11].
Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:1776307dbad63ff44654af01686051b6f997409fded66b6c4422dce43fe23d25].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 276
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 53m 34s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wa435ap555tia
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org