You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/07/23 06:41:49 UTC
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8174
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8174/display/redirect>
Changes:
------------------------------------------
[...truncated 1.18 MB...]
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config, request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 23 Jul 2021 06:41:43 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '237', '-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(3dfe8574ce53e3d7): The workflow could not be created. Causes: (3dfe8574ce53e092): A Cloud KMS key cannot be used with Streaming Engine enabled.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:603: HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:644 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:300 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720
[32mINFO [0m root:environments.py:302 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720" for Docker environment
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/requirements.txt...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/requirements.txt in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pbr-5.5.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pbr-5.5.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pbr-5.6.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pbr-5.6.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/mock-2.0.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/mock-2.0.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/six-1.16.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/six-1.16.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/soupsieve-2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/soupsieve-2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/soupsieve-2.2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/soupsieve-2.2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/PyHamcrest-1.10.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/PyHamcrest-1.10.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/parameterized-0.7.5.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/parameterized-0.7.5.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/beautifulsoup4-4.9.3.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/beautifulsoup4-4.9.3.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/dataflow_python_sdk.tar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/dataflow_python_sdk.tar in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/dataflow-worker.jar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/dataflow-worker.jar in 6 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064135-231935.1627022495.232099/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[31m[1m__________________ SideInputsTest.test_reiterable_side_input ___________________[0m
[gw7] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.transforms.sideinputs_test.SideInputsTest testMethod=test_reiterable_side_input>
[1m @pytest.mark.it_validatesrunner[0m
[1m def test_reiterable_side_input(self):[0m
[1m expected_side = frozenset(range(100))[0m
[1m [0m
[1m def check_reiteration(main, side):[0m
[1m assert expected_side == set(side), side[0m
[1m # Iterate a second time.[0m
[1m assert expected_side == set(side), side[0m
[1m # Iterate over two copies of the input at the same time.[0m
[1m both = zip(side, side)[0m
[1m first, second = zip(*both)[0m
[1m assert expected_side == set(first), first[0m
[1m assert expected_side == set(second), second[0m
[1m # This will iterate over two copies of the side input, but offset.[0m
[1m offset = [None] * (len(expected_side) // 2)[0m
[1m both = zip(itertools.chain(side, offset), itertools.chain(offset, side))[0m
[1m first, second = zip(*both)[0m
[1m expected_and_none = frozenset.union(expected_side, [None])[0m
[1m assert expected_and_none == set(first), first[0m
[1m assert expected_and_none == set(second), second[0m
[1m [0m
[1m pipeline = self.create_pipeline()[0m
[1m pcol = pipeline | 'start' >> beam.Create(['A', 'B'])[0m
[1m side = pipeline | 'side' >> beam.Create(expected_side)[0m
[1m _ = pcol | 'check' >> beam.Map(check_reiteration, beam.pvalue.AsIter(side))[0m
[1m> pipeline.run()[0m
[1m[31mapache_beam/transforms/sideinputs_test.py[0m:223:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/testing/test_pipeline.py[0m:112: in run
[1m result = super(TestPipeline, self).run([0m
[1m[31mapache_beam/pipeline.py[0m:537: in run
[1m return Pipeline.from_runner_api([0m
[1m[31mapache_beam/pipeline.py[0m:564: in run
[1m return self.runner.run_pipeline(self, self._options)[0m
[1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:53: in run_pipeline
[1m self.result = super(TestDataflowRunner,[0m
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:588: in run_pipeline
[1m self.dataflow_client.create_job(self.job), self)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:688: in create_job
[1m return self.submit_job_description(job)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:789: in submit_job_description
[1m response = self._client.projects_locations_jobs.Create(request)[0m
[1m[31mapache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py[0m:902: in Create
[1m return self._RunMethod([0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:731: in _RunMethod
[1m return self.ProcessHttpResponse(method_config, http_response, request)[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:737: in ProcessHttpResponse
[1m self.__ProcessHttpResponse(method_config, http_response, request))[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService object at 0x7f23c0462f40>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs'
method_id: 'dataflow.projects.lo...DataflowProjectsLocationsJobsCreateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 2...', request_url='https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json')
request = <DataflowProjectsLocationsJobsCreateRequest
job: <Job
environment: <Environment
clusterManagerApiService: 'compute....iles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
location: 'us-central1'
projectId: 'apache-beam-testing'>
[1m def __ProcessHttpResponse(self, method_config, http_response, request):[0m
[1m """Process the given http response."""[0m
[1m if http_response.status_code not in (http_client.OK,[0m
[1m http_client.CREATED,[0m
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config, request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 23 Jul 2021 06:41:44 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '237', '-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(9dfbf540625f1640): The workflow could not be created. Causes: (9dfbf540625f12ef): A Cloud KMS key cannot be used with Streaming Engine enabled.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:603: HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:644 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:300 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720
[32mINFO [0m root:environments.py:302 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720" for Docker environment
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/requirements.txt...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/requirements.txt in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pbr-5.5.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pbr-5.5.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pbr-5.6.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pbr-5.6.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/mock-2.0.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/mock-2.0.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/six-1.16.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/six-1.16.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/soupsieve-2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/soupsieve-2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/soupsieve-2.2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/soupsieve-2.2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/PyHamcrest-1.10.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/PyHamcrest-1.10.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/parameterized-0.7.5.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/parameterized-0.7.5.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/beautifulsoup4-4.9.3.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/beautifulsoup4-4.9.3.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/dataflow_python_sdk.tar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/dataflow_python_sdk.tar in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/dataflow-worker.jar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/dataflow-worker.jar in 7 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723064134-939196.1627022494.939476/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m========= 26 failed, 3 passed, 3 skipped, 8 warnings in 76.70 seconds ==========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 16s
92 actionable tasks: 62 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/mpl563gp7rtqc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_VR_Dataflow
#8177
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8177/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8176
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8176/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12652] revving JS filename to bust cache (#15209)
------------------------------------------
[...truncated 1.18 MB...]
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184105-465851.1627065665.466022/dataflow-worker.jar in 5 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184105-465851.1627065665.466022/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184105-465851.1627065665.466022/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[31m[1m______________ ReshuffleTest.test_reshuffle_preserves_timestamps _______________[0m
[gw3] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.transforms.util_test.ReshuffleTest testMethod=test_reshuffle_preserves_timestamps>
[1m @pytest.mark.it_validatesrunner[0m
[1m def test_reshuffle_preserves_timestamps(self):[0m
[1m with TestPipeline() as pipeline:[0m
[1m [0m
[1m # Create a PCollection and assign each element with a different timestamp.[0m
[1m before_reshuffle = ([0m
[1m pipeline[0m
[1m | beam.Create([[0m
[1m {[0m
[1m 'name': 'foo', 'timestamp': MIN_TIMESTAMP[0m
[1m },[0m
[1m {[0m
[1m 'name': 'foo', 'timestamp': 0[0m
[1m },[0m
[1m {[0m
[1m 'name': 'bar', 'timestamp': 33[0m
[1m },[0m
[1m {[0m
[1m 'name': 'bar', 'timestamp': 0[0m
[1m },[0m
[1m ])[0m
[1m | beam.Map([0m
[1m lambda element: beam.window.TimestampedValue([0m
[1m element, element['timestamp'])))[0m
[1m [0m
[1m # Reshuffle the PCollection above and assign the timestamp of an element[0m
[1m # to that element again.[0m
[1m after_reshuffle = before_reshuffle | beam.Reshuffle()[0m
[1m [0m
[1m # Given an element, emits a string which contains the timestamp and the[0m
[1m # name field of the element.[0m
[1m def format_with_timestamp(element, timestamp=beam.DoFn.TimestampParam):[0m
[1m t = str(timestamp)[0m
[1m if timestamp == MIN_TIMESTAMP:[0m
[1m t = 'MIN_TIMESTAMP'[0m
[1m elif timestamp == MAX_TIMESTAMP:[0m
[1m t = 'MAX_TIMESTAMP'[0m
[1m return '{} - {}'.format(t, element['name'])[0m
[1m [0m
[1m # Combine each element in before_reshuffle with its timestamp.[0m
[1m formatted_before_reshuffle = ([0m
[1m before_reshuffle[0m
[1m | "Get before_reshuffle timestamp" >> beam.Map(format_with_timestamp))[0m
[1m [0m
[1m # Combine each element in after_reshuffle with its timestamp.[0m
[1m formatted_after_reshuffle = ([0m
[1m after_reshuffle[0m
[1m | "Get after_reshuffle timestamp" >> beam.Map(format_with_timestamp))[0m
[1m [0m
[1m expected_data = [[0m
[1m 'MIN_TIMESTAMP - foo',[0m
[1m 'Timestamp(0) - foo',[0m
[1m 'Timestamp(33) - bar',[0m
[1m 'Timestamp(0) - bar'[0m
[1m ][0m
[1m [0m
[1m # Can't compare formatted_before_reshuffle and formatted_after_reshuffle[0m
[1m # directly, because they are deferred PCollections while equal_to only[0m
[1m # takes a concrete argument.[0m
[1m assert_that([0m
[1m formatted_before_reshuffle,[0m
[1m equal_to(expected_data),[0m
[1m label="formatted_before_reshuffle")[0m
[1m> assert_that([0m
[1m formatted_after_reshuffle,[0m
[1m equal_to(expected_data),[0m
[1m label="formatted_after_reshuffle")[0m
[1m[31mapache_beam/transforms/util_test.py[0m:611:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/pipeline.py[0m:585: in __exit__
[1m self.result = self.run()[0m
[1m[31mapache_beam/testing/test_pipeline.py[0m:112: in run
[1m result = super(TestPipeline, self).run([0m
[1m[31mapache_beam/pipeline.py[0m:537: in run
[1m return Pipeline.from_runner_api([0m
[1m[31mapache_beam/pipeline.py[0m:564: in run
[1m return self.runner.run_pipeline(self, self._options)[0m
[1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:53: in run_pipeline
[1m self.result = super(TestDataflowRunner,[0m
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:588: in run_pipeline
[1m self.dataflow_client.create_job(self.job), self)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:688: in create_job
[1m return self.submit_job_description(job)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:789: in submit_job_description
[1m response = self._client.projects_locations_jobs.Create(request)[0m
[1m[31mapache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py[0m:902: in Create
[1m return self._RunMethod([0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:731: in _RunMethod
[1m return self.ProcessHttpResponse(method_config, http_response, request)[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:737: in ProcessHttpResponse
[1m self.__ProcessHttpResponse(method_config, http_response, request))[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService object at 0x7f818c6c1310>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs'
method_id: 'dataflow.projects.lo...DataflowProjectsLocationsJobsCreateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 2...', request_url='https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json')
request = <DataflowProjectsLocationsJobsCreateRequest
job: <Job
environment: <Environment
clusterManagerApiService: 'compute....iles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
location: 'us-central1'
projectId: 'apache-beam-testing'>
[1m def __ProcessHttpResponse(self, method_config, http_response, request):[0m
[1m """Process the given http response."""[0m
[1m if http_response.status_code not in (http_client.OK,[0m
[1m http_client.CREATED,[0m
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config, request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 23 Jul 2021 18:41:13 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '237', '-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(454047edae6738d1): The workflow could not be created. Causes: (454047edae673ccc): A Cloud KMS key cannot be used with Streaming Engine enabled.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:603: HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:644 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:300 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720
[32mINFO [0m root:environments.py:302 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720" for Docker environment
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/requirements.txt...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/requirements.txt in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pbr-5.5.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pbr-5.5.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pbr-5.6.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pbr-5.6.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/mock-2.0.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/mock-2.0.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/six-1.16.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/six-1.16.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/soupsieve-2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/soupsieve-2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/soupsieve-2.2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/soupsieve-2.2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/PyHamcrest-1.10.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/PyHamcrest-1.10.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/parameterized-0.7.5.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/parameterized-0.7.5.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/beautifulsoup4-4.9.3.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/beautifulsoup4-4.9.3.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/dataflow_python_sdk.tar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/dataflow_python_sdk.tar in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/dataflow-worker.jar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/dataflow-worker.jar in 5 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723184106-060510.1627065666.060682/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m========= 26 failed, 3 passed, 3 skipped, 8 warnings in 78.00 seconds ==========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 40m 44s
92 actionable tasks: 62 executed, 30 from cache
Publishing build scan...
https://gradle.com/s/jgfqg3yk4xcqk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_VR_Dataflow #8175
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/8175/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12640] Enable adding JHM to Java based projects and add an example
------------------------------------------
[...truncated 1.18 MB...]
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125852-039787.1627045132.039952/dataflow-worker.jar in 5 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125852-039787.1627045132.039952/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125852-039787.1627045132.039952/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[31m[1m______________ ReshuffleTest.test_reshuffle_preserves_timestamps _______________[0m
[gw2] linux -- Python 3.8.5 <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8>
self = <apache_beam.transforms.util_test.ReshuffleTest testMethod=test_reshuffle_preserves_timestamps>
[1m @pytest.mark.it_validatesrunner[0m
[1m def test_reshuffle_preserves_timestamps(self):[0m
[1m with TestPipeline() as pipeline:[0m
[1m [0m
[1m # Create a PCollection and assign each element with a different timestamp.[0m
[1m before_reshuffle = ([0m
[1m pipeline[0m
[1m | beam.Create([[0m
[1m {[0m
[1m 'name': 'foo', 'timestamp': MIN_TIMESTAMP[0m
[1m },[0m
[1m {[0m
[1m 'name': 'foo', 'timestamp': 0[0m
[1m },[0m
[1m {[0m
[1m 'name': 'bar', 'timestamp': 33[0m
[1m },[0m
[1m {[0m
[1m 'name': 'bar', 'timestamp': 0[0m
[1m },[0m
[1m ])[0m
[1m | beam.Map([0m
[1m lambda element: beam.window.TimestampedValue([0m
[1m element, element['timestamp'])))[0m
[1m [0m
[1m # Reshuffle the PCollection above and assign the timestamp of an element[0m
[1m # to that element again.[0m
[1m after_reshuffle = before_reshuffle | beam.Reshuffle()[0m
[1m [0m
[1m # Given an element, emits a string which contains the timestamp and the[0m
[1m # name field of the element.[0m
[1m def format_with_timestamp(element, timestamp=beam.DoFn.TimestampParam):[0m
[1m t = str(timestamp)[0m
[1m if timestamp == MIN_TIMESTAMP:[0m
[1m t = 'MIN_TIMESTAMP'[0m
[1m elif timestamp == MAX_TIMESTAMP:[0m
[1m t = 'MAX_TIMESTAMP'[0m
[1m return '{} - {}'.format(t, element['name'])[0m
[1m [0m
[1m # Combine each element in before_reshuffle with its timestamp.[0m
[1m formatted_before_reshuffle = ([0m
[1m before_reshuffle[0m
[1m | "Get before_reshuffle timestamp" >> beam.Map(format_with_timestamp))[0m
[1m [0m
[1m # Combine each element in after_reshuffle with its timestamp.[0m
[1m formatted_after_reshuffle = ([0m
[1m after_reshuffle[0m
[1m | "Get after_reshuffle timestamp" >> beam.Map(format_with_timestamp))[0m
[1m [0m
[1m expected_data = [[0m
[1m 'MIN_TIMESTAMP - foo',[0m
[1m 'Timestamp(0) - foo',[0m
[1m 'Timestamp(33) - bar',[0m
[1m 'Timestamp(0) - bar'[0m
[1m ][0m
[1m [0m
[1m # Can't compare formatted_before_reshuffle and formatted_after_reshuffle[0m
[1m # directly, because they are deferred PCollections while equal_to only[0m
[1m # takes a concrete argument.[0m
[1m assert_that([0m
[1m formatted_before_reshuffle,[0m
[1m equal_to(expected_data),[0m
[1m label="formatted_before_reshuffle")[0m
[1m> assert_that([0m
[1m formatted_after_reshuffle,[0m
[1m equal_to(expected_data),[0m
[1m label="formatted_after_reshuffle")[0m
[1m[31mapache_beam/transforms/util_test.py[0m:611:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/pipeline.py[0m:585: in __exit__
[1m self.result = self.run()[0m
[1m[31mapache_beam/testing/test_pipeline.py[0m:112: in run
[1m result = super(TestPipeline, self).run([0m
[1m[31mapache_beam/pipeline.py[0m:537: in run
[1m return Pipeline.from_runner_api([0m
[1m[31mapache_beam/pipeline.py[0m:564: in run
[1m return self.runner.run_pipeline(self, self._options)[0m
[1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:53: in run_pipeline
[1m self.result = super(TestDataflowRunner,[0m
[1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:588: in run_pipeline
[1m self.dataflow_client.create_job(self.job), self)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:688: in create_job
[1m return self.submit_job_description(job)[0m
[1m[31mapache_beam/utils/retry.py[0m:253: in wrapper
[1m return fun(*args, **kwargs)[0m
[1m[31mapache_beam/runners/dataflow/internal/apiclient.py[0m:789: in submit_job_description
[1m response = self._client.projects_locations_jobs.Create(request)[0m
[1m[31mapache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py[0m:902: in Create
[1m return self._RunMethod([0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:731: in _RunMethod
[1m return self.ProcessHttpResponse(method_config, http_response, request)[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:737: in ProcessHttpResponse
[1m self.__ProcessHttpResponse(method_config, http_response, request))[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.dataflow.internal.clients.dataflow.dataflow_v1b3_client.DataflowV1b3.ProjectsLocationsJobsService object at 0x7fc9685bc6d0>
method_config = <ApiMethodInfo
relative_path: 'v1b3/projects/{projectId}/locations/{location}/jobs'
method_id: 'dataflow.projects.lo...DataflowProjectsLocationsJobsCreateRequest'
response_type_name: 'Job'
request_field: 'job'
supports_download: False>
http_response = Response(info={'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 2...', request_url='https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json')
request = <DataflowProjectsLocationsJobsCreateRequest
job: <Job
environment: <Environment
clusterManagerApiService: 'compute....iles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
location: 'us-central1'
projectId: 'apache-beam-testing'>
[1m def __ProcessHttpResponse(self, method_config, http_response, request):[0m
[1m """Process the given http response."""[0m
[1m if http_response.status_code not in (http_client.OK,[0m
[1m http_client.CREATED,[0m
[1m http_client.NO_CONTENT):[0m
[1m> raise exceptions.HttpError.FromResponse([0m
[1m http_response, method_config=method_config, request=request)[0m
[1m[31mE apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 23 Jul 2021 12:59:02 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '237', '-content-encoding': 'gzip'}>, content <{[0m
[1m[31mE "error": {[0m
[1m[31mE "code": 400,[0m
[1m[31mE "message": "(983af44b3a88e684): The workflow could not be created. Causes: (983af44b3a88eab3): A Cloud KMS key cannot be used with Streaming Engine enabled.",[0m
[1m[31mE "status": "FAILED_PRECONDITION"[0m
[1m[31mE }[0m
[1m[31mE }[0m
[1m[31mE >[0m
[1m[31m../../build/gradleenv/-1734967051/lib/python3.8/site-packages/apitools/base/py/base_api.py[0m:603: HttpBadRequestError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:644 Executing command: ['<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/bin/python3.8',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:300 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python SDK docker image has Python 3.8 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.8_sdk:2.33.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720
[32mINFO [0m root:environments.py:302 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python38-fnapi:beam-master-20210720" for Docker environment
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/requirements.txt...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/requirements.txt in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pbr-5.5.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pbr-5.5.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pbr-5.6.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pbr-5.6.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/mock-2.0.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/mock-2.0.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/six-1.16.0.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/six-1.16.0.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/soupsieve-2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/soupsieve-2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/soupsieve-2.2.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/soupsieve-2.2.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/PyHamcrest-1.10.1.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/PyHamcrest-1.10.1.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/parameterized-0.7.5.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/parameterized-0.7.5.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/beautifulsoup4-4.9.3.tar.gz...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/beautifulsoup4-4.9.3.tar.gz in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/dataflow_python_sdk.tar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/dataflow_python_sdk.tar in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/dataflow-worker.jar...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/dataflow-worker.jar in 5 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:632 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:648 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0723125854-477331.1627045134.477492/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/-1734967051/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py38-xdist.xml> -
[31m[1m========= 26 failed, 3 passed, 3 skipped, 8 warnings in 72.17 seconds ==========[0m
> Task :sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 189
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesRunnerStreamingTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 58m 34s
92 actionable tasks: 82 executed, 10 from cache
Publishing build scan...
https://gradle.com/s/n4ttpb3jbpbbi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org