You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/10/27 06:42:42 UTC
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #1366
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1366/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12978] Customizable dependency for Java external transform
[heejong] add ConfigT param to getDependencies method
------------------------------------------
[...truncated 716.99 KB...]
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed. Causes: Job appears to be stuck. Several workers have failed to start up in a row, and no worker has successfully started up for this job. Last error reported: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image..
apache_beam/runners/dataflow/dataflow_runner.py:1643: DataflowRuntimeException
------------------------------ Captured log call -------------------------------
INFO apache_beam.runners.portability.stager:stager.py:303 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.35.0.dev
INFO root:environments.py:296 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20211015
INFO root:environments.py:304 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20211015" for Docker environment
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fd5a426c6a8> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fd5a426ce18> ====================
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:454 Defaulting to the temp_location as staging_location: gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/beam-sdks-java-testing-expansion-service-testExpansionService-2.35.0-SNAPSHOT-O43zcCUfdR3oTjmYYqWM94AknVWsxLmVJ8HCKDFiMlY.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/beam-sdks-java-testing-expansion-service-testExpansionService-2.35.0-SNAPSHOT-O43zcCUfdR3oTjmYYqWM94AknVWsxLmVJ8HCKDFiMlY.jar in 3 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/dataflow_python_sdk.tar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/dataflow_python_sdk.tar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:638 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/pipeline.pb...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:657 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-1027063230-770302.1635316350.770851/pipeline.pb in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:818 Create job: <Job
clientRequestId: '20211027063230771790-3474'
createTime: '2021-10-27T06:32:36.246307Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-10-26_23_32_35-17138742594254128411'
location: 'us-central1'
name: 'beamapp-jenkins-1027063230-770302'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-10-27T06:32:36.246307Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:820 Created job with id: [2021-10-26_23_32_35-17138742594254128411]
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:821 Submitted job: 2021-10-26_23_32_35-17138742594254128411
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:827 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-26_23_32_35-17138742594254128411?project=apache-beam-testing
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-26_23_32_35-17138742594254128411 is in state JOB_STATE_RUNNING
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:38.902Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2021-10-26_23_32_35-17138742594254128411. The number of workers will be between 1 and 1000.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:39.099Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2021-10-26_23_32_35-17138742594254128411.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:40.875Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.634Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.665Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.730Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.767Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/CoGroupByKeyImpl/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.797Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.833Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.863Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.902Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.928Z: JOB_MESSAGE_DEBUG: Inserted coder converter before flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.962Z: JOB_MESSAGE_DEBUG: Inserted coder converter before flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:41.995Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Flatten_27 for input ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_25.None-post16
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.028Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write, through flatten assert_that/Group/CoGroupByKeyImpl/Flatten, into producer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.064Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values) into assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.086Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/RestoreTags into assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.119Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/RestoreTags
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.151Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.184Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write into assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.218Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/FlatMap(<lambda at core.py:3222>) into Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.273Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/AddRandomKeys into Create/FlatMap(<lambda at core.py:3222>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.307Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into Create/MaybeReshuffle/Reshuffle/AddRandomKeys
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.345Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.369Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.396Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.464Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys into Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.500Z: JOB_MESSAGE_DETAILED: Fusing consumer Create/Map(decode) into Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.531Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:prefix)/Map/ParMultiDo(Anonymous) into Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.568Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into ExternalTransform(beam:transforms:xlang:test:prefix)/Map/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.601Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/FlatMap(<lambda at core.py:3222>) into assert_that/Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.634Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/Map(decode) into assert_that/Create/FlatMap(<lambda at core.py:3222>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.665Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Tag[0] into assert_that/Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.691Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity into assert_that/Group/CoGroupByKeyImpl/Tag[0]
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.711Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.744Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Tag[1] into assert_that/ToVoidKey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.768Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity into assert_that/Group/CoGroupByKeyImpl/Tag[1]
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.805Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.823Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.847Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:42.868Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.012Z: JOB_MESSAGE_DEBUG: Executing wait step start26
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.082Z: JOB_MESSAGE_BASIC: Executing operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.117Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.130Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.163Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.216Z: JOB_MESSAGE_BASIC: Finished operation Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.229Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.297Z: JOB_MESSAGE_DEBUG: Value "Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.332Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/CoGroupByKeyImpl/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.364Z: JOB_MESSAGE_BASIC: Executing operation Create/Impulse+Create/FlatMap(<lambda at core.py:3222>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:43.401Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:3222>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:32:52.560Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:33:23.730Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:34:07.994Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:34:33.673Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:35:15.756Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:35:42.108Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:36:26.879Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:36:50.146Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:37:36.899Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:03.587Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.611Z: JOB_MESSAGE_WARNING: A worker was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.647Z: JOB_MESSAGE_ERROR: Workflow failed. Causes: Job appears to be stuck. Several workers have failed to start up in a row, and no worker has successfully started up for this job. Last error reported: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: manifest for us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111 not found: manifest unknown: Failed to fetch "20211027060111" from request "/v2/apache-beam-testing/java-postcommit-it/java/manifests/20211027060111".. This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow workers have permissions to pull image..
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.695Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:3222>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/InputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.695Z: JOB_MESSAGE_BASIC: Finished operation Create/Impulse+Create/FlatMap(<lambda at core.py:3222>)+Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.764Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.829Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:48.847Z: JOB_MESSAGE_BASIC: Stopping worker pool...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:38:51.047Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 1 to 0.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:41:50.131Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-10-27T06:41:50.168Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-10-26_23_32_35-17138742594254128411 is in state JOB_STATE_FAILED
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/pytest_xlangValidateRunner.xml> -
============== 9 failed, 1 passed, 10 warnings in 1132.05 seconds ==============
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava FAILED
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 20063.
Stopping expansion service pid: 20067.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ddb8a6c8b5fb6217ec0477770f96270bd5ff604a22a0feafd492e942acc556a6
Deleted: sha256:1400ae5ed00208bee11abca50eba277df25e7502c4f0f6d66243993fb0e662ba
Deleted: sha256:d3e12eaf8fda5591173f84a26f8b1bf64ad25281e997ae224952c28f61abf5c4
Deleted: sha256:1c6944777a8394ec1022f14219aabfe8e971381aabe834ccc6208e554d09c217
Deleted: sha256:b3fece3ebd6be67a502a0e5d3a4665a24b0d107bf26fd0fa36d565c446dbd419
Deleted: sha256:c0da1ab06cb23794d29bd40c452bc71faf1b0e2ded24a38935d3062edececdb2
Deleted: sha256:919ffe08323c4ee86d9c788f1f4625fda068fd394358ba2f886849f2f23bb0ad
Deleted: sha256:04b653bb4afb04e496d672f266c07a80da80748cbbd579e2e12acad266a41893
Deleted: sha256:3abb5cfaec79897c5e0445894632e7cdd3964cf177c5994ef50190df4c7bf71e
Deleted: sha256:85cb8eaa25ee971b66ae2151f5702521fe979391e29d5b92f25e299cece40706
Deleted: sha256:44e265edc283dfbef1e99f9eb98957bca43ee29977b39e33f381996e11926a38
Deleted: sha256:7867857c2499ca3bc5ec67700c5c25d3958e19002164080fa6c319b4d2bc8e5c
Deleted: sha256:8c815ecc824b1d7a910abbc22106396bf6b4cd3574592b93b4ac96b4c0a15f5a
Deleted: sha256:6ac75c14e1abe171f6e5e054916432e307f4757e899949cae74c9b8e8cd86b93
Deleted: sha256:3cb53c9476b82e7048d4f330067874760d30cbe3227afd611f55f0161cd64d0f
Deleted: sha256:3c67db24f69a88e14a56242fc054503a42c59500cc974a91a3bf5e3aff6d218d
Deleted: sha256:f26c424cd345d6b43ec8df23fc7bfb4367837d7c9be1ef486cc024c542e6a604
Deleted: sha256:a331e3c624a85b86f55f54d99c749a3aca254e9d94b9438e395f1e86a2eee3ce
Deleted: sha256:cf92f384aff5fcd85227a08d3c6483cfc79ce399040494aa00600c340e8ebd5f
Deleted: sha256:2987b5ca1094d3a2d0e2bf56340b0fd9061b0127550c84f63c70d8b6429eb22e
Deleted: sha256:b22acc822e460bd308d6ce0f7329e50eac0a0a4e878a207b57f29543a9f143f9
Deleted: sha256:d7cc592a3cfb827d821dc06cfa4b1d8f2e7ea810bd0d6f357c54d08496decdfd
Deleted: sha256:85e90e1b2a8d6d9636379534d5d1112cfa7dc3a536e61349b7951291b3c9e11b
Deleted: sha256:045b120b9fc13fb16c1eb8980595bcf503966d88734d99e17753967e663df10b
Deleted: sha256:87afac0c8a3bac9cb5c107acd32ff5cfdc0a7de050a81be221070e6dd2b1c299
Deleted: sha256:ce19ea10602cba8e1f28807fbfb464a97d4e1b3e7b20e99e5b49e79a9a1d1dce
Deleted: sha256:aaa5e31e44ccea9e28f8b4457cb9bda7282694570fe3e73b47a9d366a7f5a1e1
Deleted: sha256:bbae8e73e142256ba954d3c358d5a5caad1478581b08d9d8c21afc172425974b
Deleted: sha256:51abac25b6f70ce3ede052a83bddce51aeab6172eb1cd520843bd979b1e0ed75
ERROR: (gcloud.container.images.untag) Image could not be found: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027060111]
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 278
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 42m 3s
132 actionable tasks: 112 executed, 16 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xd4dzm3vpv4f6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_XVR_Dataflow
#1368
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1368/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #1367
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/1367/display/redirect?page=changes>
Changes:
[mmack] [BEAM-8958] Use AWS credentials provider with BasicKinesisProvider (AWS
------------------------------------------
[...truncated 457.75 KB...]
google-cloud-bigquery-storage==2.9.1
google-cloud-bigtable==1.0.0
google-cloud-core==1.4.1
google-cloud-datastore==1.15.3
google-cloud-dlp==0.13.0
google-cloud-language==1.3.0
google-cloud-profiler==3.0.4
google-cloud-pubsub==1.0.2
google-cloud-recommendations-ai==0.2.0
google-cloud-spanner==1.13.0
google-cloud-videointelligence==1.13.0
google-cloud-vision==0.42.0
google-crc32c==1.3.0
google-pasta==0.2.0
google-python-cloud-debugger==2.15
google-resumable-media==1.3.3
googleapis-common-protos==1.53.0
grpc-google-iam-v1==0.12.3
grpcio==1.40.0
grpcio-gcp==0.2.2
guppy3==3.0.10
h5py==3.1.0
hdfs==2.5.8
httplib2==0.19.1
idna==2.10
importlib-metadata==4.8.1
joblib==1.1.0
keras==2.6.0
Keras-Preprocessing==1.1.2
libcst==0.3.21
Markdown==3.3.4
mmh3==2.5.1
more-itertools==8.10.0
mypy-extensions==0.4.3
nltk==3.5
numpy==1.19.5
oauth2client==4.1.3
oauthlib==3.1.1
opt-einsum==3.3.0
orjson==3.6.1
packaging==21.0
pandas==1.1.5
Pillow==7.2.0
pip==21.2.4
pluggy==0.13.1
proto-plus==1.19.6
protobuf==3.17.3
protorpc==0.12.0
PTable==0.9.2
py==1.10.0
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pydot==1.4.1
PyHamcrest==1.10.1
pymongo==3.10.1
pyparsing==2.4.7
pytest==4.6.11
python-dateutil==2.8.1
python-gflags==3.1.2
python-snappy==0.5.4
pytz==2020.1
PyYAML==5.4
regex==2021.10.23
requests==2.24.0
requests-oauthlib==1.3.0
rsa==4.7.2
scikit-learn==0.24.1
scipy==1.4.1
setuptools==57.5.0
six==1.15.0
soupsieve==2.2.1
tenacity==8.0.1
tensorboard==2.7.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.6.0
tensorflow-estimator==2.6.0
termcolor==1.1.0
threadpoolctl==3.0.0
tqdm==4.62.3
typing-extensions==3.7.4.3
typing-inspect==0.7.1
uritemplate==4.1.1
urllib3==1.25.11
wcwidth==0.2.5
Werkzeug==2.0.2
wheel==0.37.0
wrapt==1.12.1
zipp==3.6.0
Removing intermediate container 92903abf44d1
---> d7cb15fd5796
Step 22/27 : RUN pip check
---> Running in 7bfa4edd0f57
No broken requirements found.
Removing intermediate container 7bfa4edd0f57
---> cd13ed29405e
Step 23/27 : COPY target/LICENSE /opt/apache/beam/
---> bb4ae385b78c
Step 24/27 : COPY target/LICENSE.python /opt/apache/beam/
---> 5e5cf1478ee3
Step 25/27 : COPY target/NOTICE /opt/apache/beam/
---> 1ce084c46006
Step 26/27 : ADD target/launcher/linux_amd64/boot /opt/apache/beam/
---> ddae3238bdd2
Step 27/27 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in ecf662e53b75
Removing intermediate container ecf662e53b75
---> 0a1026346df8
Successfully built 0a1026346df8
Successfully tagged apache/beam_python3.6_sdk:2.35.0.dev
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup
Launching Java expansion service @ 46047
Launching Python expansion service @ 34935
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --sdk_harness_container_image_overrides=.*java.*,us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027120111 --experiments=use_runner_v2 --experiments=shuffle_mode=appliance --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz>
>>> pytest options: --capture=no --numprocesses=8 --timeout=4500 --log-cli-level=INFO
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[gw2] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw1] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw0] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw4] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw7] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw3] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw6] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
[gw5] Python 3.6.8 (default, Dec 24 2018, 19:24:27) -- [GCC 5.4.0 20160609]
gw0 [10] / gw1 [10] / gw2 [10] / gw3 [10] / gw4 [10] / gw5 [10] / gw6 [10] / gw7 [10]
scheduling tests via LoadScheduling
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
[gw6] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
[gw0] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
[gw7] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
[gw4] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
[gw1] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_prefix
[gw2] PASSED apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_partition
[gw3] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
[gw5] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
[gw1] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_prefix
[gw2] PASSED apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_partition
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/pytest_xlangValidateRunner.xml> -
=================== 10 passed, 4 warnings in 1315.66 seconds ===================
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 8072.
Stopping expansion service pid: 8088.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027120111
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:818597130e09914bba0be4b7f4e4f830fbc14de686e1dd3d71770f1fc84f4e70
Deleted: sha256:77720ee365d206b01db9976a11ced6e8225e878af22f7058bffa989eff20b65e
Deleted: sha256:223f59b04947adaf0b3288e1639c41ca17f1881d026c6815dba875aa3c36c05b
Deleted: sha256:dba60d022f4c50de41e83b0a38be9fb2eca22e0012f0d920eaa25005db737cfb
Deleted: sha256:2b71140d4bf7d74729c57374e89b5d3612d54bd4ea837b7e625c3827f82315b2
Deleted: sha256:4c3116778ff84647b93e856c9eb4914bca69b1bec594a9ca5196d2011eb795b2
Deleted: sha256:917aedb2accf052874cca76c013af5270478ba380abe38fc557fcdf19da1b7c3
Deleted: sha256:6a5c2a825b9457058cdd10211f222e58f0b87878bc8308851036463e7061ae15
Deleted: sha256:043f67a5287537c2021aa16a3d2584b031896e34448b3790f27b5a4c736170d7
Deleted: sha256:9ee239bc309d722b68ebf38ac536efa737fe5c0ae0f911b2600a12661b78dffd
Deleted: sha256:03c177752799437f42e1827c4d18dc4b85826b596b91cd2ec82117241c4b7e80
Deleted: sha256:4ace7d2ac57969e1fa6c5380f71ca1a78505dc1ed9e1d6e62299fcbc9767d782
Deleted: sha256:68a4ecffb231031376dfbe1649d14efc9b6e7986f78591ebfbfeda01fc9ee292
Deleted: sha256:9ffc247b8dc24daf0aa67f175ce483633fa930ef0349a38ef910517f7c334bab
Deleted: sha256:1eb70b132324f771ca2d3be559eb9723f7beb2d8043866fcc6e0e36ede5dfd56
Deleted: sha256:d2b2556adb71dea7f9bcbd2b410da7ffc13154f899871a7c8bd51aed0e508e90
Deleted: sha256:ef05b7af1ba8adc88286e97df8dbff94b73b3fa9d28ee4a286d59f949304a9ed
Deleted: sha256:c9f0c525ef9d0c074ca95fffaec3c10be7bd52741fde555e0613f18008971d3e
Deleted: sha256:6c0ca70f69943b2b40a9fe9d82336d3d4807640a01f764685e70fa6406d5ce76
Deleted: sha256:4eb10e6445d77e167bf1f5155cd994ba8296857c549b5121b79850738822a35c
Deleted: sha256:719821eee87ae803c7c77f5480e8582bbfddfae5b9fde639d748f45ad07684cd
Deleted: sha256:c97cd655443e0e9667a29b047877080b2ab83555da18fbc5109a8a0ee205086e
Deleted: sha256:c35b184c36dd18b6c02df94432ca6045a509acb86d2bbeb741ec55f61a03195a
Deleted: sha256:0af9fc37017956935ebf0acb8efca0673d2e8f016e4850305f9780dc8a2f0ad7
Deleted: sha256:7fd51fa20d08941bdd8ff9b4965a833ceb8c27c7af80decb12f7a540af426cba
Deleted: sha256:d78bc940d8f8f999e528ab99645da1311a5311420112dddb443fcd101a895b95
Deleted: sha256:4bd3f5895ccb8f5d2dff9f7cd60055756ab9dcddcb3cff38393efd085c500cb6
Deleted: sha256:2d630cf7414067171aaa7f4643b46a03e6da7a16dec262039a698a772714cb27
Deleted: sha256:ab25b17c0235247ef45197855b01ff6923738c0fe5c127bbc9809966017c9082
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027120111]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:818597130e09914bba0be4b7f4e4f830fbc14de686e1dd3d71770f1fc84f4e70]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211027120111] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:818597130e09914bba0be4b7f4e4f830fbc14de686e1dd3d71770f1fc84f4e70])].
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770eca01651ff2e303427bd3d4087fa3ddcffac935ff4bb9bee3c4b9175e689d
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:770eca01651ff2e303427bd3d4087fa3ddcffac935ff4bb9bee3c4b9175e689d
ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Wed, 27 Oct 2021 12:42:07 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'}
Failed to compute blob liveness for manifest: 'sha256:770eca01651ff2e303427bd3d4087fa3ddcffac935ff4bb9bee3c4b9175e689d': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 281
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 34s
132 actionable tasks: 98 executed, 30 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/dyth3ufjjlzew
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org