You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/06/25 18:33:06 UTC
Build failed in Jenkins: beam_PostCommit_XVR_Dataflow #874
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/874/display/redirect?page=changes>
Changes:
[noreply] [BEAM-9547] Add implementation for from_dict, from_records (#15034)
[noreply] [BEAM-11951] added "Differences from Pandas" page for DataFrame (#15074)
[noreply] [BEAM-9547] reindex is order-sensitive (#15032)
------------------------------------------
[...truncated 1.23 MB...]
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:51.396Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:51.438Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-06-25_11_31_29-12476500062621846372 is in state JOB_STATE_FAILED
________ ValidateRunnerXlangTest.test_multi_input_output_with_sideinput ________
[gw3] linux -- Python 3.6.8 <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/build/gradleenv/1922375555/bin/python3.6>
self = <apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest testMethod=test_multi_input_output_with_sideinput>
test_pipeline = None
@pytest.mark.xlang_transforms
def test_multi_input_output_with_sideinput(self, test_pipeline=None):
CrossLanguageTestPipelines().run_multi_input_output_with_sideinput(
> test_pipeline or self.create_pipeline())
apache_beam/transforms/validate_runner_xlang_test.py:259:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/transforms/validate_runner_xlang_test.py:121: in run_multi_input_output_with_sideinput
assert_that(res['side'], equal_to(['ss']), label='CheckSide')
apache_beam/pipeline.py:585: in __exit__
self.result = self.run()
apache_beam/testing/test_pipeline.py:114: in run
False if self.not_use_test_runner_api else test_runner_api))
apache_beam/pipeline.py:564: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
createTime: '2021-06-25T18:31:31.008704Z'
currentState: CurrentStateValueValuesEnum(JOB...021-06-25T18:31:31.008704Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f96f9195fd0>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1629: DataflowRuntimeException
------------------------------ Captured log call -------------------------------
INFO apache_beam.runners.portability.stager:stager.py:300 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.32.0.dev
INFO root:environments.py:296 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20210622
INFO root:environments.py:304 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python36-fnapi:beam-master-20210622" for Docker environment
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:650 ==================== <function pack_combiners at 0x7f96fb43b2f0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:650 ==================== <function sort_stages at 0x7f96fb43ba60> ====================
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:452 Defaulting to the temp_location as staging_location: gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/icedtea-sound-Bdoi2wYa757-fzq5vconCy4SSQ22ZaOq7yuC98fKPs8.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/jaccess-CMbK-IOdQPLKHEqCuDnE-yBk-VpbtVT-hgjbHRUGO78.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/localedata-ae5Z0L6ak4922fztWeWy7ajiWXdG3ubNrwerJRFoFj0.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/nashorn-XHtz_UehGpYcLTOrATrTnMNVUgEVa_ttoWkPxnVfqTo.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/cldrdata-k07I6K9W3X5KTQbcDIEsqM0LXyM18f0eR6IaJw-P_xk.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/dnsns-RGhCDg3GVOQVC2r6ka2N0hmI4eqQH6VobuoAnQ74MnE.jar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/beam-sdks-java-testing-expansion-service-testExpansionService-2.32.0-SNAPSHOT-r2gF5NoKrv2Y2yzSA5y9TNf74qdrZxzSVFwgQYg5t7M.jar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/beam-sdks-java-testing-expansion-service-testExpansionService-2.32.0-SNAPSHOT-r2gF5NoKrv2Y2yzSA5y9TNf74qdrZxzSVFwgQYg5t7M.jar in 5 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/dataflow_python_sdk.tar...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/dataflow_python_sdk.tar in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/pipeline.pb...
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:651 Completed GCS upload to gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0625183122-142941.1624645882.143741/pipeline.pb in 0 seconds.
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:798 Create job: <Job
createTime: '2021-06-25T18:31:31.008704Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-06-25_11_31_29-1061865775003673254'
location: 'us-central1'
name: 'beamapp-jenkins-0625183122-142941'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-06-25T18:31:31.008704Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:800 Created job with id: [2021-06-25_11_31_29-1061865775003673254]
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:801 Submitted job: 2021-06-25_11_31_29-1061865775003673254
INFO apache_beam.runners.dataflow.internal.apiclient:apiclient.py:807 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-06-25_11_31_29-1061865775003673254?project=apache-beam-testing
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-06-25_11_31_29-1061865775003673254 is in state JOB_STATE_RUNNING
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:33.251Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2021-06-25_11_31_29-1061865775003673254. The number of workers will be between 1 and 1000.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:33.398Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2021-06-25_11_31_29-1061865775003673254.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:35.667Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.224Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.263Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.339Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.374Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.404Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step CheckSide/Group/GroupByKey: GroupByKey not followed by a combiner.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.497Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.536Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.588Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.625Z: JOB_MESSAGE_DEBUG: Inserted coder converter after flatten ref_AppliedPTransform_CheckSide-Group-Flatten_45
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.657Z: JOB_MESSAGE_DEBUG: Inserted coder converter after flatten ref_AppliedPTransform_assert_that-Group-Flatten_29
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.684Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_assert_that-Group-Flatten_29 for input ref_AppliedPTransform_assert_that-Group-pair_with_0_27.None
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.719Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/Flatten/OutputIdentity, through flatten assert_that/Group/Flatten, into producer assert_that/Group/pair_with_0
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.741Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/Map(_merge_tagged_vals_under_key) into assert_that/Group/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.773Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/Map(_merge_tagged_vals_under_key)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.810Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.844Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_assert_that-Group-Flatten_29-u32 for input ref_AppliedPTransform_assert_that-Group-Flatten_29.None-c30
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.877Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Write, through flatten assert_that/Group/Flatten/Unzipped-1, into producer assert_that/Group/Flatten/OutputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.917Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/Flatten/OutputIdentity into assert_that/Group/pair_with_1
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.956Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Write into assert_that/Group/Flatten/OutputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:36.987Z: JOB_MESSAGE_DETAILED: Unzipping flatten external_1ExternalTransform-beam-transforms-xlang-test-multi--Flatten-PCollections for input ref_AppliedPTransform_Main1-Map-decode-_6.None
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.027Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of ExternalTransform(beam:transforms:xlang:test:multi)/ParDo(Anonymous)/ParMultiDo(Anonymous), through flatten ExternalTransform(beam:transforms:xlang:test:multi)/Flatten.PCollections, into producer Main1/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.059Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_CheckSide-Group-Flatten_45 for input ref_AppliedPTransform_CheckSide-Group-pair_with_0_43.None
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.107Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of CheckSide/Group/Flatten/OutputIdentity, through flatten CheckSide/Group/Flatten, into producer CheckSide/Group/pair_with_0
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.142Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Group/Map(_merge_tagged_vals_under_key) into CheckSide/Group/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.176Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Unkey into CheckSide/Group/Map(_merge_tagged_vals_under_key)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.198Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Match into CheckSide/Unkey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.226Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.262Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_1 into assert_that/ToVoidKey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.294Z: JOB_MESSAGE_DETAILED: Unzipping flatten external_1ExternalTransform-beam-transforms-xlang-test-multi--Flatten-PCollections-u39 for input external_1ExternalTransform-beam-transforms-xlang-test-multi--ParDo-Anonymous--ParMultiDo-Anonymous-.output-c37
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.328Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/WindowInto(WindowIntoFn), through flatten ExternalTransform(beam:transforms:xlang:test:multi)/Flatten.PCollections/Unzipped-1, into producer ExternalTransform(beam:transforms:xlang:test:multi)/ParDo(Anonymous)/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.364Z: JOB_MESSAGE_DETAILED: Unzipping flatten ref_AppliedPTransform_CheckSide-Group-Flatten_45-u44 for input ref_AppliedPTransform_CheckSide-Group-Flatten_45.None-c42
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.395Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of CheckSide/Group/GroupByKey/Write, through flatten CheckSide/Group/Flatten/Unzipped-1, into producer CheckSide/Group/Flatten/OutputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.447Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/MapElements/Map/ParMultiDo(Anonymous) into Side/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.483Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/WithKeys/AddKeys/Map/ParMultiDo(Anonymous) into Side/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.504Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/ParDo(Anonymous)/ParMultiDo(Anonymous) into Main2/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.537Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Group/Flatten/OutputIdentity into CheckSide/Group/pair_with_1
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.575Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into ExternalTransform(beam:transforms:xlang:test:multi)/ParDo(Anonymous)/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.610Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Group/GroupByKey/Write into CheckSide/Group/Flatten/OutputIdentity
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.643Z: JOB_MESSAGE_DETAILED: Fusing consumer Side/FlatMap(<lambda at core.py:2972>) into Side/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.676Z: JOB_MESSAGE_DETAILED: Fusing consumer Side/Map(decode) into Side/FlatMap(<lambda at core.py:2972>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.697Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/WithKeys/AddKeys/Map/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.728Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Write into ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.757Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues into ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Read
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.786Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.817Z: JOB_MESSAGE_DETAILED: Fusing consumer ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Values/Values/Map/ParMultiDo(Anonymous) into ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.851Z: JOB_MESSAGE_DETAILED: Fusing consumer Main1/FlatMap(<lambda at core.py:2972>) into Main1/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.886Z: JOB_MESSAGE_DETAILED: Fusing consumer Main1/Map(decode) into Main1/FlatMap(<lambda at core.py:2972>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.920Z: JOB_MESSAGE_DETAILED: Fusing consumer Main2/FlatMap(<lambda at core.py:2972>) into Main2/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.945Z: JOB_MESSAGE_DETAILED: Fusing consumer Main2/Map(decode) into Main2/FlatMap(<lambda at core.py:2972>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:37.977Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/WindowInto(WindowIntoFn) into ExternalTransform(beam:transforms:xlang:test:multi)/MapElements/Map/ParMultiDo(Anonymous)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.007Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Create/FlatMap(<lambda at core.py:2972>) into CheckSide/Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.041Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Create/Map(decode) into CheckSide/Create/FlatMap(<lambda at core.py:2972>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.065Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Group/pair_with_0 into CheckSide/Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.111Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/ToVoidKey into CheckSide/WindowInto(WindowIntoFn)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.137Z: JOB_MESSAGE_DETAILED: Fusing consumer CheckSide/Group/pair_with_1 into CheckSide/ToVoidKey
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.161Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/FlatMap(<lambda at core.py:2972>) into assert_that/Create/Impulse
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.200Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Create/Map(decode) into assert_that/Create/FlatMap(<lambda at core.py:2972>)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.232Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_0 into assert_that/Create/Map(decode)
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.272Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.309Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.343Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.375Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.537Z: JOB_MESSAGE_DEBUG: Executing wait step start67
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.606Z: JOB_MESSAGE_BASIC: Executing operation ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.640Z: JOB_MESSAGE_BASIC: Executing operation CheckSide/Group/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.654Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.673Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.683Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.727Z: JOB_MESSAGE_BASIC: Finished operation ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.739Z: JOB_MESSAGE_BASIC: Finished operation CheckSide/Group/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.739Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/GroupByKey/Create
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.800Z: JOB_MESSAGE_DEBUG: Value "ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.827Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.855Z: JOB_MESSAGE_DEBUG: Value "CheckSide/Group/GroupByKey/Session" materialized.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.884Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:2972>)+assert_that/Create/Map(decode)+assert_that/Group/pair_with_0+assert_that/Group/Flatten/OutputIdentity+assert_that/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.945Z: JOB_MESSAGE_BASIC: Executing operation Side/Impulse+Side/FlatMap(<lambda at core.py:2972>)+Side/Map(decode)+ExternalTransform(beam:transforms:xlang:test:multi)/MapElements/Map/ParMultiDo(Anonymous)+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/WithKeys/AddKeys/Map/ParMultiDo(Anonymous)+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Write+CheckSide/WindowInto(WindowIntoFn)+CheckSide/ToVoidKey+CheckSide/Group/pair_with_1+CheckSide/Group/Flatten/OutputIdentity+CheckSide/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:38.977Z: JOB_MESSAGE_BASIC: Executing operation CheckSide/Create/Impulse+CheckSide/Create/FlatMap(<lambda at core.py:2972>)+CheckSide/Create/Map(decode)+CheckSide/Group/pair_with_0+CheckSide/Group/Flatten/OutputIdentity+CheckSide/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:31:58.941Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.064Z: JOB_MESSAGE_ERROR: Startup of the worker pool in zone us-central1-b failed to bring up any of the desired 1 workers. ZONE_RESOURCE_POOL_EXHAUSTED: The zone 'projects/apache-beam-testing/zones/us-central1-b' does not have enough resources available to fulfill the request. Try a different zone, or try again later.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.094Z: JOB_MESSAGE_ERROR: Workflow failed.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.165Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at core.py:2972>)+assert_that/Create/Map(decode)+assert_that/Group/pair_with_0+assert_that/Group/Flatten/OutputIdentity+assert_that/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.165Z: JOB_MESSAGE_BASIC: Finished operation Side/Impulse+Side/FlatMap(<lambda at core.py:2972>)+Side/Map(decode)+ExternalTransform(beam:transforms:xlang:test:multi)/MapElements/Map/ParMultiDo(Anonymous)+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/WithKeys/AddKeys/Map/ParMultiDo(Anonymous)+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+ExternalTransform(beam:transforms:xlang:test:multi)/View.AsSingleton/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/GroupByKey/Write+CheckSide/WindowInto(WindowIntoFn)+CheckSide/ToVoidKey+CheckSide/Group/pair_with_1+CheckSide/Group/Flatten/OutputIdentity+CheckSide/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.165Z: JOB_MESSAGE_BASIC: Finished operation CheckSide/Create/Impulse+CheckSide/Create/FlatMap(<lambda at core.py:2972>)+CheckSide/Create/Map(decode)+CheckSide/Group/pair_with_0+CheckSide/Group/Flatten/OutputIdentity+CheckSide/Group/GroupByKey/Write
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.244Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.315Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:27.342Z: JOB_MESSAGE_BASIC: Stopping worker pool...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:49.824Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236 2021-06-25T18:32:49.863Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job 2021-06-25_11_31_29-1061865775003673254 is in state JOB_STATE_FAILED
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/ws/src/sdks/python/pytest_xlangValidateRunner.xml> -
========================== 8 failed in 131.65 seconds ==========================
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava FAILED
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 26291.
Stopping expansion service pid: 26295.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625182329
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e63bb731a6673bd8150ebcb90295ee4948933f82ab779aecaa619740601448f6
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e63bb731a6673bd8150ebcb90295ee4948933f82ab779aecaa619740601448f6
Associated tags:
- 20210625182329
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625182329
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210625182329].
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:e63bb731a6673bd8150ebcb90295ee4948933f82ab779aecaa619740601448f6].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 38s
103 actionable tasks: 29 executed, 74 up-to-date
Publishing build scan...
https://gradle.com/s/afmzdy36pqc2k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_XVR_Dataflow #875
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/875/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org