You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/14 15:28:21 UTC
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 343 -
Aborted!
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 343 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/343/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #502
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/502/display/redirect>
Changes:
------------------------------------------
[...truncated 324.23 KB...]
Using cached google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB)
Collecting google-cloud-pubsub<3,>=2.1.0
Using cached google_cloud_pubsub-2.9.0-py2.py3-none-any.whl (219 kB)
Collecting google-cloud-pubsublite<2,>=1.2.0
Using cached google_cloud_pubsublite-1.4.0-py2.py3-none-any.whl (261 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.34.0-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-bigquery-storage>=2.6.3
Using cached google_cloud_bigquery_storage-2.11.0-py2.py3-none-any.whl (172 kB)
Collecting google-cloud-core<2,>=0.28.1
Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.0-py2.py3-none-any.whl (111 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.4-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.4
Using cached botocore-1.24.4-py3-none-any.whl (8.5 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675864 sha256=7485ed5a9e41108235021d9e687d87d6b5d738292049f81aa0051229cba3dbd4
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.4 botocore-1.24.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe90d151c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe90d152440> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0222102058.1645542117.345849/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0222102058.1645542117.345849/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0222102058.1645542117.345849/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0222102058.1645542117.345849/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220222150157346799-4991'
createTime: '2022-02-22T15:01:58.479364Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-22_07_01_57-14080043857411804753'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0222102058'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-22T15:01:58.479364Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-22_07_01_57-14080043857411804753]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-22_07_01_57-14080043857411804753
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-22_07_01_57-14080043857411804753?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-22_07_01_57-14080043857411804753 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:01.327Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.170Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.213Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.248Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.279Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.329Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.374Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.415Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.480Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.512Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.555Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.595Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.631Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.675Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.696Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.738Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.875Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.942Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:02.989Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:03.018Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-22_07_01_57-14080043857411804753 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:36.330Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 7 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:36.357Z: JOB_MESSAGE_DETAILED: Resized **** pool to 7, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:37.714Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:02:46.575Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:03:08.009Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:03:08.041Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:12.972Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.058Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.091Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.169Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.222Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.291Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:09:13.344Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy121.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #586
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/586/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13015] Update the SDK harness grouping table to be memory bounded
[noreply] [BEAM-13982] Added output of logging for python E2E pytests (#17637)
[noreply] [BEAM-14473] Throw error if using globally windowed, unbounded side
[noreply] [BEAM-14440] Add basic fuzz tests to the coders package (#17587)
[noreply] [BEAM-14035 ] Implement BigQuerySchema Read/Write TransformProvider
[noreply] Add Akvelon to case-studies (#17611)
[noreply] Merge pull request #17520 from BEAM-12356 Close DatasetService leaked
[noreply] Adding eslint and lint configuration to TypeScript SDK (#17676)
[noreply] Update release-guide.md
[noreply] Update release-guide.md
[noreply] [BEAM-14411] Re-enable TypecodersTest, fix most issues (#17547)
[noreply] Merge pull request #17678 from [BEAM-14460] [Playground] WIP. Fix error
------------------------------------------
[...truncated 329.35 KB...]
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.3.1)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2776974 sha256=82df37f504bf12ee0d94d26c7677e98177e8dfb8f52f7ef78f8388b17215f237
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.23.1 botocore-1.26.1 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.1 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.4 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ffa90107e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ffa90106680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0517100510.1652802904.470451/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0517100510.1652802904.470451/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0517100510.1652802904.470451/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0517100510.1652802904.470451/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220517155504471861-6359'
createTime: '2022-05-17T15:55:05.708475Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-17_08_55_05-5019473855042774752'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0517100510'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-17T15:55:05.708475Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-17_08_55_05-5019473855042774752]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-17_08_55_05-5019473855042774752
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-17_08_55_05-5019473855042774752?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-17_08_55_05-5019473855042774752 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:09.189Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:10.641Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:10.846Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.021Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.050Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.104Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.138Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.185Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.238Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.267Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.305Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.329Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.376Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.413Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.446Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.482Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.673Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.762Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.823Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:11.862Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-17_08_55_05-5019473855042774752 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:19.435Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:55:35.214Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T15:56:10.291Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:04.621Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:04.733Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:04.775Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:04.849Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:04.910Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:05.001Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T16:03:05.066Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:00:52.635Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-17_08_55_05-5019473855042774752.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:00:52.688Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-17_08_55_05-5019473855042774752 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:00:52.804Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:00:53.230Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:00:53.254Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:01:35.180Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:01:35.222Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-17T20:01:35.248Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-17_08_55_05-5019473855042774752 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-17_08_55_05-5019473855042774752?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 7m 57s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
Publishing failed.
The build scan server appears to be unavailable.
Please check https://status.gradle.com for the latest service status.
If the service is reported as available, please report this problem via https://gradle.com/help/plugin and include the following via copy/paste:
----------
Gradle version: 7.4
Plugin version: 3.4.1
Request URL: https://status.gradle.com
Request ID: 98ea2449-f176-4e6e-877c-de4227dfd3fb
Response status code: 405
Response server type: Varnish
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #585
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/585/display/redirect>
Changes:
------------------------------------------
[...truncated 328.64 KB...]
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.27.0,>=1.26.0
Using cached botocore-1.26.0-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2777028 sha256=2a147f8ee8d5600482d312734293b0df21ebd53a41d41aaa6b8641615c1a9f34
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.23.0 botocore-1.26.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.1 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.4 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f21a6d229e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f21a6d24200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0516100416.1652714625.844294/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0516100416.1652714625.844294/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0516100416.1652714625.844294/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0516100416.1652714625.844294/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220516152345845272-4873'
createTime: '2022-05-16T15:23:46.839913Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-16_08_23_46-3157782307257617332'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0516100416'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-16T15:23:46.839913Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-16_08_23_46-3157782307257617332]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-16_08_23_46-3157782307257617332
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-16_08_23_46-3157782307257617332?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-16_08_23_46-3157782307257617332 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:48.730Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.639Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.669Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.722Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.750Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.787Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.813Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.874Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.907Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:49.996Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.064Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.095Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.129Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.165Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.291Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.422Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.457Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:23:50.486Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-16_08_23_46-3157782307257617332 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:24:13.067Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:24:24.644Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:24:46.370Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.154Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.213Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.245Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.345Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.395Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.461Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T15:30:25.515Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:00:45.296Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-16_08_23_46-3157782307257617332.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:00:45.318Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:00:45.491Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:00:45.524Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:00:45.540Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-16_08_23_46-3157782307257617332 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:01:19.927Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:01:19.961Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-16T20:01:19.993Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-16_08_23_46-3157782307257617332 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-16_08_23_46-3157782307257617332?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 38m 53s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/3zddelj3nsvxe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #584
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/584/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14470] Use Generic Registrations in loadtests. (#17673)
------------------------------------------
[...truncated 327.94 KB...]
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.23.0-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.27.0,>=1.26.0
Using cached botocore-1.26.0-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2777028 sha256=857d0b403fba2be07794f04ab5e5818526a3979cad5b57383f1d6298427a2a4d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.23.0 botocore-1.26.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.1 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.4 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f38538299e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f385382a200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0515100431.1652630227.134334/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0515100431.1652630227.134334/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0515100431.1652630227.134334/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0515100431.1652630227.134334/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220515155707135305-6643'
createTime: '2022-05-15T15:57:08.049483Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-15_08_57_07-752468079755972449'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0515100431'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-15T15:57:08.049483Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-15_08_57_07-752468079755972449]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-15_08_57_07-752468079755972449
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-15_08_57_07-752468079755972449?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-15_08_57_07-752468079755972449 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:10.267Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:11.650Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:11.786Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:11.831Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:11.867Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.016Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.072Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.134Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.192Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.284Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.333Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.425Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.460Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.507Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.532Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.646Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.733Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.815Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:12.849Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-15_08_57_07-752468079755972449 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:35.145Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:57:42.038Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T15:58:08.550Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.380Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.483Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.505Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.575Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.648Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.717Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T16:04:52.774Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:00:53.924Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-15_08_57_07-752468079755972449.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:00:53.948Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:00:54.022Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:00:54.198Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:00:54.218Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-15_08_57_07-752468079755972449 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:01:25.693Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:01:25.730Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-15T20:01:25.754Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-15_08_57_07-752468079755972449 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-15_08_57_07-752468079755972449?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 5m 32s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ts23nhdng7ayc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #583
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/583/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14455] Add UUID to sub-schemas for PythonExternalTransform
[Heejong Lee] [BEAM-14430] Adding a logical type support for Python callables to Row
[Heejong Lee] add urn, type inference for PythonCallableSource
[Heejong Lee] fix lint errors
[Heejong Lee] move logical types def
[Heejong Lee] add micros_instant urn
[Heejong Lee] put a default type hint for PythonCallableSource
[Heejong Lee] add comment
[noreply] Revert "Better test assertion. (#17551)"
[noreply] Bump github.com/spf13/cobra from 1.3.0 to 1.4.0 in /sdks (#17647)
[noreply] [BEAM-14465] Reduce DefaultS3ClientBuilderFactory logging to debug level
[noreply] Merge pull request #17365 from [BEAM-12482] Update Schema Destination
[noreply] [BEAM-14014] Support impersonation credentials in dataflow runner
[noreply] [BEAM-14469] Allow nil primary returns from TrySplit in a single-window
[noreply] Add some auto-starting runners to the typescript SDK. (#17580)
[noreply] [BEAM-14371] (and BEAM-14372) - enable a couple staticchecks (#17670)
------------------------------------------
[...truncated 328.21 KB...]
Using cached SQLAlchemy-1.4.36-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.4-py2.py3-none-any.whl (39 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.27.0,>=1.26.0
Using cached botocore-1.26.0-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2777028 sha256=6dac0c1fc40532e09ca0586997dfae07747b6e1a55e9b788ca02c43867399b40
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.23.0 botocore-1.26.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.1 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.4 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f599ed319e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f599ed32200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0514100413.1652543730.848988/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0514100413.1652543730.848988/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0514100413.1652543730.848988/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0514100413.1652543730.848988/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220514155530849937-9999'
createTime: '2022-05-14T15:55:31.712259Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-14_08_55_31-1481361689829031140'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0514100413'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-14T15:55:31.712259Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-14_08_55_31-1481361689829031140]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-14_08_55_31-1481361689829031140
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-14_08_55_31-1481361689829031140?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-14_08_55_31-1481361689829031140 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:33.721Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.390Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.432Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.463Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.514Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.546Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.576Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.605Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.637Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.697Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.737Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.763Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.859Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.919Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:34.973Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:35.044Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:35.206Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:35.286Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:35.352Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:35.394Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-14_08_55_31-1481361689829031140 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:55.733Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:55:57.959Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T15:56:30.044Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.581Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.677Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.708Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.772Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.825Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:14.922Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T16:03:15.001Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:00:41.036Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-14_08_55_31-1481361689829031140.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:00:41.095Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:00:41.255Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:00:41.299Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:00:41.390Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-14_08_55_31-1481361689829031140 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:01:15.693Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:01:15.725Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-14T20:01:15.779Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-14_08_55_31-1481361689829031140 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-14_08_55_31-1481361689829031140?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 56s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/c6domqtmavyh2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #582
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/582/display/redirect?page=changes>
Changes:
[dannymccormick] [BEAM-14441] Add GitHub issue templates
[dannymccormick] Ask for beam version + other dependencies
[dannymccormick] We don't need outage
[dannymccormick] Cut p4
[chamikaramj] Updates CHANGES.md to include some recently discovered known issues
[noreply] [BEAM-14334] Fix leakage of SparkContext in Spark runner tests to remove
[noreply] Typo & link update (#17633)
[dannymccormick] Pare down to fewer templates
[noreply] Trigger go precommits on go mod/sum changes (#17636)
[noreply] Revert "[BEAM-14429] Force java load test on dataflow runner v2
[noreply] [BEAM-14347] Add generic registration feature to CHANGES (#17643)
[noreply] Better test assertion. (#17551)
[noreply] Bump github.com/google/go-cmp from 0.5.7 to 0.5.8 in /sdks (#17628)
[noreply] Bump github.com/testcontainers/testcontainers-go in /sdks (#17627)
[noreply] Bump github.com/lib/pq from 1.10.4 to 1.10.5 in /sdks (#17626)
[noreply] Merge pull request #17584 from [BEAM-14415] Exception handling tests and
[noreply] Bump cloud.google.com/go/pubsub from 1.18.0 to 1.21.1 in /sdks (#17646)
[noreply] Merge pull request #17408 from [BEAM-14312] [Website] change section
[noreply] Bump cloud.google.com/go/bigquery from 1.28.0 to 1.32.0 in /sdks
[noreply] [BEAM-14347] Add function for simple function registration (#17650)
[noreply] Drop dataclasses requirement, we only support python 3.7+ (#17640)
------------------------------------------
[...truncated 329.46 KB...]
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.12.0-py3-none-any.whl (366 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.0-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.13
Using cached botocore-1.25.13-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.1-py2.py3-none-any.whl (211 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2773552 sha256=4ba0c7d46acdc1bc61e8188d345995a703630678da6885735951d6c6ed4db898
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.22.13 botocore-1.25.13 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.1 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2ee8384320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2ee8384b00> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0513100445.1652456647.625782/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0513100445.1652456647.625782/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0513100445.1652456647.625782/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0513100445.1652456647.625782/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220513154407627013-3232'
createTime: '2022-05-13T15:44:09.044727Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-13_08_44_08-17450588225137531890'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0513100445'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-13T15:44:09.044727Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-13_08_44_08-17450588225137531890]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-13_08_44_08-17450588225137531890
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-13_08_44_08-17450588225137531890?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-13_08_44_08-17450588225137531890 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:33.286Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:38.673Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:38.806Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:38.857Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:38.912Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.026Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.081Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.150Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.194Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.227Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.257Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.284Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.311Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.348Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.422Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:39.485Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-13_08_44_08-17450588225137531890 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:40.737Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:41.400Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:41.695Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:41.817Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:44:44.838Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:45:16.733Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:45:41.784Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.426Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.484Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.508Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.604Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.654Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.716Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T15:52:20.775Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:00:41.791Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-13_08_44_08-17450588225137531890.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:00:42.168Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:00:42.488Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:00:42.563Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:00:42.590Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-13_08_44_08-17450588225137531890 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:01:17.797Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:01:18.408Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-13T20:01:18.565Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-13_08_44_08-17450588225137531890 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-13_08_44_08-17450588225137531890?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 18m 59s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/qay27rdwredrw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #581
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/581/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-14096] bump junit-quickcheck to 1.0
[noreply] [BEAM-5492] Python Dataflow integration tests should export the pipeline
[noreply] [BEAM-14396] Bump httplib2 upper bound. (#17602)
[noreply] [BEAM-11104] Add self-checkpointing to CHANGES.md (#17612)
[noreply] [BEAM-14081] [CdapIO] Add context classes for CDAP plugins (#17104)
[noreply] [BEAM-12526] Add Dependabot (#17563)
[noreply] Remove python 3.6 postcommit from mass_comment.py (#17630)
[noreply] [BEAM-14347] Add some benchmarks for generic registration (#17613)
[noreply] Correctly route go dependency changes to go label (#17632)
[noreply] [BEAM-13695] Add jamm jvm options to Java 11 (#17178)
------------------------------------------
[...truncated 329.07 KB...]
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.12.0-py3-none-any.whl (366 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.0-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.12
Using cached botocore-1.25.12-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.1-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.40.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2771561 sha256=fe0c31ca953802746fc670ab44974d7bff445e72bfa65f455bece43d41413fb0
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.22.12 botocore-1.25.12 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.1 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4f2a36d290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4f2a36da70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0512100431.1652370716.096521/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0512100431.1652370716.096521/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0512100431.1652370716.096521/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0512100431.1652370716.096521/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220512155156097440-4582'
createTime: '2022-05-12T15:51:57.230040Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-12_08_51_56-17769454670639241342'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0512100431'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-12T15:51:57.230040Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-12_08_51_56-17769454670639241342]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-12_08_51_56-17769454670639241342
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-12_08_51_56-17769454670639241342?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-12_08_51_56-17769454670639241342 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:00.146Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.050Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.087Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.115Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.187Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.226Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.274Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.308Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.378Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.447Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.534Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.618Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:01.914Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.028Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.133Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.225Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-12_08_51_56-17769454670639241342 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.467Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.535Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.582Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:02.604Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:25.382Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:38.752Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:52:58.969Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.468Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.519Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.546Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.606Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.654Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.710Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T15:59:42.774Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:00:39.373Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-12_08_51_56-17769454670639241342.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:00:39.400Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:00:42.912Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:00:43.076Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:00:43.113Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-12_08_51_56-17769454670639241342 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:01:20.948Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:01:20.997Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-12T20:01:21.023Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-12_08_51_56-17769454670639241342 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-05-12_08_51_56-17769454670639241342?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1675, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 10m 36s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/zks4et5eehyfc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #580
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/580/display/redirect?page=changes>
Changes:
[Alexey Romanenko] [BEAM-12918] Add PostCommit_Java_Tpcds_Spark job
[johnjcasey] [BEAM-14448] add datastore test
[yathu] [BEAM-14423] Add test cases for BigtableIO.BigtableWriterFn fails due to
[Pablo Estrada] Revert "Merge pull request #17517 from [BEAM-14383] Improve "FailedRows"
[noreply] [BEAM-14229] Fix SyntheticUnboundedSource duplication from checkpoint
[noreply] [BEAM-14347] Rename registration package to register (#17603)
[noreply] [BEAM-11104] Add self-checkpointing integration test (#17590)
------------------------------------------
[...truncated 329.79 KB...]
Using cached boto3-1.22.11-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.26.0,>=1.25.11
Using cached botocore-1.25.11-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2771396 sha256=a811fd0ff812424df545446cec059dcc115584fe513a58242e38eae19a54596b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.9
Uninstalling pyparsing-3.0.9:
Successfully uninstalled pyparsing-3.0.9
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.12.0 boto3-1.22.11 botocore-1.25.11 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f21ade698c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f21ade6a0e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0511100418.1652284424.766509/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0511100418.1652284424.766509/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0511100418.1652284424.766509/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0511100418.1652284424.766509/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220511155344767453-1661'
createTime: '2022-05-11T15:53:45.811472Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-11_08_53_45-2872587195487053483'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0511100418'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-11T15:53:45.811472Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-11_08_53_45-2872587195487053483]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-11_08_53_45-2872587195487053483
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-11_08_53_45-2872587195487053483?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-11_08_53_45-2872587195487053483 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:47.720Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.298Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.327Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.355Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.376Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.407Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.452Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.474Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.499Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.518Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.544Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.568Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.585Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.608Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.666Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.770Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.820Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.849Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:53:48.868Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-11_08_53_45-2872587195487053483 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:54:11.133Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:54:11.150Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:54:18.790Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:54:21.337Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T15:54:43.753Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:03Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:03.344Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:03.683Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:03.989Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:04.112Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:04.190Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T16:00:04.274Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:00:40.067Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-11_08_53_45-2872587195487053483.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:00:40.093Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:00:40.339Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:00:40.412Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:00:40.443Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-11_08_53_45-2872587195487053483 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:01:14.243Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:01:14.285Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-11T20:01:14.306Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-11_08_53_45-2872587195487053483 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 8m 46s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/de5pu6oci65nm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #579
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/579/display/redirect?page=changes>
Changes:
[andyye333] Add extra details to PubSub matcher errors
[noreply] Merge pull request #17559 from [BEAM-14423] Add exception injection
[noreply] [BEAM-11104] Allow self-checkpointing SDFs to return without finishing
[noreply] Merge pull request #17544 from [BEAM-14415] Exception handling tests for
[noreply] Merge pull request #17565 from [BEAM-14413] add Kafka exception test
[noreply] Merge pull request #17555 from [BEAM-14417] Adding exception handling
[noreply] [BEAM-14433] Improve Go split error message. (#17575)
[noreply] [BEAM-14429] Force java load test on dataflow runner v2
[noreply] Merge pull request #17577 from [BEAM-14435] Adding exception handling
[noreply] [BEAM-14347] Add generic registration functions for iters and emitters
[noreply] [BEAM-14169] Add Credentials rotation cron job for clusters (#17383)
[noreply] [BEAM-14347] Add generic registration for accumulators (#17579)
------------------------------------------
[...truncated 329.37 KB...]
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.10
Using cached botocore-1.25.10-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2771942 sha256=9d98bd1bdd8c9f2cf23184fbf3603d4b8129dc9383aa2e6b2875f113d9b03e2d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.11.0 boto3-1.22.10 botocore-1.25.10 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3b1511f170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3b1511f950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0510100444.1652197663.217501/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0510100444.1652197663.217501/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0510100444.1652197663.217501/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0510100444.1652197663.217501/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220510154743218778-1850'
createTime: '2022-05-10T15:47:44.339849Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-10_08_47_43-12573554164792080462'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0510100444'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-10T15:47:44.339849Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-10_08_47_43-12573554164792080462]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-10_08_47_43-12573554164792080462
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-10_08_47_43-12573554164792080462?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-10_08_47_43-12573554164792080462 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:46.969Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.111Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.151Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.215Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.241Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.283Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.317Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.351Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.382Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.415Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.437Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.480Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.517Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.566Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.597Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.629Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.763Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.841Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.918Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:47:48.949Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-10_08_47_43-12573554164792080462 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:48:03.508Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:48:11.265Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:48:43.518Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:23.943Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.013Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.045Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.108Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.175Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.246Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T15:55:24.338Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:00:43.145Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-10_08_47_43-12573554164792080462.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:00:43.174Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:00:43.333Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:00:43.411Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:00:43.435Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-10_08_47_43-12573554164792080462 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:01:24.760Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:01:24.809Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-10T20:01:24.838Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-10_08_47_43-12573554164792080462 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 15m 2s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/23vfqyicyqq6s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #578
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/578/display/redirect?page=changes>
Changes:
[elias.segundo] Changing elegibility to AllNodeElegibility
[chamikaramj] Adds code reviewers for GCP I/O connectors and KafkaIO to Beam OWNERS
------------------------------------------
[...truncated 328.59 KB...]
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.26.0,>=1.25.9
Using cached botocore-1.25.9-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2770580 sha256=a5ea0e2d61265da097a0eb73d0a21aedffdf4ec59c954dcb314e5c93ba980a70
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.11.0 boto3-1.22.9 botocore-1.25.9 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd0b9af6200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd0b9af69e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0509100417.1652111232.656234/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0509100417.1652111232.656234/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0509100417.1652111232.656234/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0509100417.1652111232.656234/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220509154712657166-5456'
createTime: '2022-05-09T15:47:13.629691Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-09_08_47_13-12598048265630042825'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0509100417'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-09T15:47:13.629691Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-09_08_47_13-12598048265630042825]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-09_08_47_13-12598048265630042825
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-09_08_47_13-12598048265630042825?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-09_08_47_13-12598048265630042825 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:17.891Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.243Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.273Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.300Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.392Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.426Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.452Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.506Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.534Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.560Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.586Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.608Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.639Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.663Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.697Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.730Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.909Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:19.964Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:20.018Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:20.051Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-09_08_47_13-12598048265630042825 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:42.245Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:42.279Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:42.482Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:47:52.462Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:48:14.753Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.018Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.110Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.136Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.179Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.276Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.336Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T15:53:50.394Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-09_08_47_13-12598048265630042825 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:00:40.320Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-09_08_47_13-12598048265630042825.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:00:40.344Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:00:43.567Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:00:43.779Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:00:43.806Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:01:17.193Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:01:17.231Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-09T20:01:17.253Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-09_08_47_13-12598048265630042825 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 15m 18s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ejvzoozskvdb6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #577
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/577/display/redirect>
Changes:
------------------------------------------
[...truncated 328.96 KB...]
Using cached cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.9
Using cached botocore-1.25.9-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2769970 sha256=64aa88b216bbbd319c758cc7e081a2c3500d01185ce4e29438961b84dfa667e6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.11.0 boto3-1.22.9 botocore-1.25.9 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f618ea0e200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f618ea0e9e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0508100427.1652024924.679018/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0508100427.1652024924.679018/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0508100427.1652024924.679018/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0508100427.1652024924.679018/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220508154844679952-6931'
createTime: '2022-05-08T15:48:45.727694Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-08_08_48_45-5825726509403692619'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0508100427'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-08T15:48:45.727694Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-08_08_48_45-5825726509403692619]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-08_08_48_45-5825726509403692619
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-08_08_48_45-5825726509403692619?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-08_08_48_45-5825726509403692619 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:48.610Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:50.822Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:50.862Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:50.906Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:50.942Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.015Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.041Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.075Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.109Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.142Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.168Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.201Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.240Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.272Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.307Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.341Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.470Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.539Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.585Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:48:51.657Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-08_08_48_45-5825726509403692619 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:49:14.184Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:49:14.216Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:49:21.302Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:49:24.421Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:49:47.990Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.492Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.560Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.593Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.660Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.712Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.772Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T15:56:02.848Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:00:37.873Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-08_08_48_45-5825726509403692619.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:00:37.902Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:00:38.707Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:00:38.752Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:00:38.811Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-08_08_48_45-5825726509403692619 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:01:10.700Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:01:10.745Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-08T20:01:10.770Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-08_08_48_45-5825726509403692619 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 13m 37s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/tsgbb226drwgc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #576
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/576/display/redirect?page=changes>
Changes:
[kevinsijo] Setting up a basic directory
[kevinsijo] Mirroring Python SDK's directory structure
[kerrydc] Adds initial tests
[kevinsijo] 'runners' is the correct directory name
[Pablo Estrada] sketching the core API for JS SDK
[jonathanlui] add .gitignore for node/ts project
[Robert Bradshaw] Worker directory.
[Robert Bradshaw] Fix complile errors with explicit any for callables.
[Robert Bradshaw] Add worker entry point.
[Robert Bradshaw] Add proto generation code.
[Robert Bradshaw] Add generated proto files.
[Robert Bradshaw] Attempts to get ts protos to compile.
[Robert Bradshaw] Exclude ts protos for now.
[Robert Bradshaw] More changes to get ts protos working.
[Robert Bradshaw] Update scripts and config to get protos compiling.
[Robert Bradshaw] Update geenrated files.
[jonathanlui] add build and clean script to compile ts
[Robert Bradshaw] Generate server for loopback worker.
[Robert Bradshaw] Generated grpc servers for loopback.
[Robert Bradshaw] Add typescript formatter.
[Robert Bradshaw] Loopback server (that does nothing).
[Robert Bradshaw] Working server.
[Pablo Estrada] Starting expansion of primitive transforms
[Pablo Estrada] Starting to implement and support standard coders
[Robert Bradshaw] Also generate grpc clients.
[Robert Bradshaw] Basic implementation of worker harness.
[Pablo Estrada] fix the build
[Robert Bradshaw] Add some missing files for worker harness.
[Robert Bradshaw] Refactor operators to use registration.
[jonathanlui] enable ts in mocha
[jonathanlui] update readme
[jonathanlui] --save-dev @types/mocha
[jonathanlui] translate core_test.js to typescript
[Robert Bradshaw] Encapsulate worker service in a class.
[Kenneth Knowles] Port standard_coders_test to typescript (superficially)
[Pablo Estrada] Starting the proto translation of Impulse, ParDo, GBK
[Robert Bradshaw] Add some tests for the worker code.
[Robert Bradshaw] Fixing old lock file error.
[Pablo Estrada] Adding transform names and fixing GBK coder issue
[Robert Bradshaw] npx tsfmt -r src/apache_beam/base.ts src/apache_beam/transforms/core.ts
[Kenneth Knowles] switch to import style require() statements
[Kenneth Knowles] Add Coder interface using protobufjs classes
[Kenneth Knowles] BytesCoder with some failures
[noreply] Added GeneralObjectCoder and using it as coder for most transforms (#9)
[Kenneth Knowles] Fix order of arguments to deepEqual
[Kenneth Knowles] Encode expected encoding as binary
[Robert Bradshaw] Refactor API to allow for composites.
[jrmccluskey] Initial setup for automated Java expansion startup
[jrmccluskey] Update exp_service.ts
[Kenneth Knowles] Fix up coder deserialization
[Robert Bradshaw] Simplify GBK coder computation.
[Robert Bradshaw] Remove top-level PValue.
[Pablo Estrada] Make tests green
[Robert Bradshaw] Rename PValueish to PValue.
[jonathanlui] node runner
[jonathanlui] whitespaces
[Robert Bradshaw] Make Runner.run async.
[jonathanlui] bson and fast-deep-equal should not be listed as devdependency
[jrmccluskey] Add basic Dockerfile that starts ExternalWorkerPool
[Robert Bradshaw] Direct runner.
[kevinsijo] Testing expansion service communication
[Robert Bradshaw] Added flatten, assertion checkers.
[Pablo Estrada] progress on basic coders
[Robert Bradshaw] Fixing the build.
[Robert Bradshaw] Cleanup, simplify access.
[Pablo Estrada] Adding limited support for KVCoder and IterableCoder
[Robert Bradshaw] Introduce PipelineContext.
[Robert Bradshaw] Add toProto to all coders.
[Robert Bradshaw] Some work with coders.
[Robert Bradshaw] Remove debug logging.
[Robert Bradshaw] Use coders over data channel.
[Kenneth Knowles] explicitly sequence sub-coder serializations
[Kenneth Knowles] no more need to extend FakeCoder
[Kenneth Knowles] actually advance reader
[Kenneth Knowles] autoformat
[Kenneth Knowles] protobufjs already can write and read signed varints
[Kenneth Knowles] with improved test harness, kv has many more failures
[Kenneth Knowles] read bytescoder from correct position
[Kenneth Knowles] no more fake coders
[Kenneth Knowles] varint examples all work
[Kenneth Knowles] simplify coder value parsing
[Kenneth Knowles] global window coder
[Kenneth Knowles] fix swapEndian32
[Robert Bradshaw] Add P(...) operator.
[kevinsijo] Implementing RowCoder encoding.
[jrmccluskey] remove unused container dir
[kevinsijo] Corrected sorting of encoded positions to reflect an argsort instead.
[Robert Bradshaw] Populate environments.
[kevinsijo] Implementing RowCoder decoding.
[Kenneth Knowles] preliminary unbounded iterable coder
[Kenneth Knowles] friendlier description of standard coder test case
[Kenneth Knowles] fix test harness; iterable works
[jrmccluskey] first pass at boot.go
[jonathanlui] update package-lock.json
[jonathanlui] make NodeRunner a subclass of Runner
[jonathanlui] add waitUntilFinish interface member
[Pablo Estrada] Adding double coder
[Kenneth Knowles] scaffolding for windowed values
[Pablo Estrada] Adding type information to PColleciton and PTransform
[jonathanlui] fix direct runner
[Pablo Estrada] Adding typing information for DoFns
[Kenneth Knowles] add interval window
[Robert Bradshaw] Export PValue.
[Robert Bradshaw] Add CombineFn interface.
[Robert Bradshaw] Typed flatten.
[jonathanlui] add runAsync method to base.Runner
[Kenneth Knowles] add Long package
[Pablo Estrada] Adding more types. Making PValue typed
[Kenneth Knowles] instant coder draft
[Robert Bradshaw] Return job state from direct runner.
[Kenneth Knowles] type instant = long
[jonathanlui] implement NodeRunner.runPipeline
[Kenneth Knowles] autoformat
[kevinsijo] Completed implementation of basic row coder
[Kenneth Knowles] Fix IntervalWindowCoder, almost
[Kenneth Knowles] fix interval window coder
[Kenneth Knowles] autoformat
[Robert Bradshaw] loopback runner works
[Kenneth Knowles] move core element types into values.ts
[Kenneth Knowles] just build object directly to be cool
[Robert Bradshaw] GBK working on ULR.
[Robert Bradshaw] Async transforms.
[Robert Bradshaw] External transform grpah splicing.
[Kenneth Knowles] progress on windowed value: paneinfo encoding
[Robert Bradshaw] Fix merge.
[Robert Bradshaw] autoformat
[Kenneth Knowles] full windowed value coder
[kerrydc] Updates tests to use correct types, adds generics where needed to DoFns
[Robert Bradshaw] Add serialization librarires.'
[Robert Bradshaw] Add Split() PTransform, for producing multiple outputs from a single
[Robert Bradshaw] Schema-encoded external payloads.
[kevinsijo] Adding Schema inference from JSON
[Pablo Estrada] Removing unused directories
[Pablo Estrada] Support for finishBundle and improving typing annotations.
[Pablo Estrada] A base implementation of combiners with GBK/ParDo
[Robert Bradshaw] Fully propagate windowing information in both remote and direct runner.
[Robert Bradshaw] Make args and kwargs optional for python external transform.
[Robert Bradshaw] Infer schema for external transforms.
[Pablo Estrada] Implementing a custom combine fn as an example. Small fixes
[Robert Bradshaw] Fix missing windowing information in combiners.
[Robert Bradshaw] PostShuffle needn't group by key as that's already done.
[Robert Bradshaw] Guard pre-combine for global window only.
[Robert Bradshaw] WindowInto
[Robert Bradshaw] Fix optional kwargs.
[Robert Bradshaw] A couple of tweaks for js + py
[Robert Bradshaw] Add windowing file.
[Robert Bradshaw] CombineBy transform, stand-alone WordCount.
[Robert Bradshaw] cleanup
[Robert Bradshaw] Actually fix optional external kwargs.
[Robert Bradshaw] Demo2, textio read.
[Robert Bradshaw] Add command lines for starting up the servers.
[Robert Bradshaw] Run prettier on the full codebase.
[Robert Bradshaw] Update deps.
[Pablo Estrada] Adding docstrings for core.ts. Prettier dependency
[Pablo Estrada] Documenting coder interfaces
[Pablo Estrada] Added documentation for a few standard coders
[Robert Bradshaw] Unified grouping and combining.
[Robert Bradshaw] Allow PCollection ids to be lazy.
[Robert Bradshaw] Reorganize module structure.
[Robert Bradshaw] A couple more renames.
[Robert Bradshaw] Simplify.
[Robert Bradshaw] Consolidation.
[Robert Bradshaw] Fix build.
[Robert Bradshaw] Add optional context to ParDo.
[Robert Bradshaw] fixup: iterable coder endian sign issue
[Robert Bradshaw] omit context for map(console.log)
[Robert Bradshaw] Fix ReadFromText coders.
[Robert Bradshaw] Flesh out README with overview and current state.
[noreply] Readme typo
[Robert Bradshaw] Two more TODOs.
[noreply] Add a pointer to the example wordcount to the readme.
[Pablo Estrada] Documenting coders and implementing unknown-length method
[Robert Bradshaw] UIID dependency.
[Robert Bradshaw] Artifact handling.
[Robert Bradshaw] Properly wait on data channel for bundle completion.
[Robert Bradshaw] Automatic java expansion service startup.
[Robert Bradshaw] Process promises.
[Robert Bradshaw] Implement side inputs.
[Robert Bradshaw] Cleanup.
[Robert Bradshaw] Put complex constext stuff in its own file.
[Robert Bradshaw] Rename BoundedWindow to just Window.
[Robert Bradshaw] Alternative splitter class.
[Pablo Estrada] Documenting internal functions
[Robert Bradshaw] Take a pass clarifying the TODOs.
[Robert Bradshaw] Sql transform wrapper.
[Robert Bradshaw] Incorporate some feedback into the TODOs.
[Robert Bradshaw] More TODOs.
[Robert Bradshaw] Remove app placeholder.
[Robert Bradshaw] Apache license headers.
[Robert Bradshaw] More TODOs
[jankuehle] Suggestions for TypeScript todos
[dannymccormick] Add actions for typescript sdk
[dannymccormick] Fix test command
[noreply] Add missing version
[dannymccormick] Fix codecovTest command
[noreply] Only do prettier check on linux
[noreply] Only get codecov on linux
[Robert Bradshaw] Resolve some comments.
[Robert Bradshaw] Fix compile errors.
[Robert Bradshaw] Prettier.
[Robert Bradshaw] Re-order expandInternal arguments pending unification.
[Robert Bradshaw] More consistent and stricter PTransform naming.
[Robert Bradshaw] Notes on explicit, if less idiomatic, use of classes.
[Robert Bradshaw] Let DoFn be an interface rather than a class.
[Robert Bradshaw] Provide DoFn context to start and finish bundle.
[Robert Bradshaw] Optional promise code simplification.
[Robert Bradshaw] Cleanup todos.
[Robert Bradshaw] Avoid any type where not needed.
[Robert Bradshaw] Apache RAT excludes for typescript.
[Robert Bradshaw] Remove empty READMEs.
[Robert Bradshaw] Add licences statement to readme files.
[Robert Bradshaw] More RAT fixes.
[Robert Bradshaw] Another unsupported coder.
[Robert Bradshaw] Remove debugging code.
[noreply] Fix automatic naming with code coverage.
[Robert Bradshaw] Coders cleanup.
[Robert Bradshaw] Add tests for RowCoder.
[Robert Bradshaw] Normalize capitalization, comments.
[Robert Bradshaw] Install typescript closure packages.
[Robert Bradshaw] npm audit fix
[Robert Bradshaw] Move more imports out of base.
[Robert Bradshaw] Changes needed to compile with ts closure plugin.
[Robert Bradshaw] Use ttsc and ts-closure-transform plugin.
[Robert Bradshaw] Serialization registration to actually get serialization working.
[Robert Bradshaw] Container images working on local runner.
[Robert Bradshaw] Add a portable job server that proxies the Dataflow backend. (#17189)
[Robert Bradshaw] Improvements to dataflow job service for non-Python jobs.
[Robert Bradshaw] Get dataflow working.
[Robert Bradshaw] User friendly pipeline options.
[Robert Bradshaw] Less classes, more functions.
[Robert Bradshaw] Add new nullable standard coder.
[Robert Bradshaw] Make Apache Rat happy.
[Robert Bradshaw] Disable broken codecov.
[Robert Bradshaw] Remove last uses of base.ts.
[Robert Bradshaw] Remove unneedd file.
[Robert Bradshaw] Remove more uneeded/unused files.
[Robert Bradshaw] Cleanup tests.
[Robert Bradshaw] Minor cleanups to coder tests.
[noreply] Quote pip install package name
[noreply] [BEAM-14374] Fix module import error in FullyQualifiedNamedTransform
[Robert Bradshaw] Addressing issues from the review.
[noreply] Apply suggestions from code review.
[Robert Bradshaw] Post-merge fixes.
[dannymccormick] Delete tags.go
[Robert Bradshaw] Update tests to use our actual serialization libraries.
[Robert Bradshaw] Another pass at TODOs, removing finished items.
[Heejong Lee] [BEAM-14146] Python Streaming job failing to drain with BigQueryIO write
[Kenneth Knowles] Add parameter for service account impersonation in GCP credentials
[Heejong Lee] add test
[noreply] Merge pull request #17490 from [BEAM-14370] [Website] Add new page about
[noreply] [BEAM-14332] Refactored cluster management for Flink on Dataproc
[noreply] [BEAM-13988] Update mtime to use time.UnixMilli() calls (#17578)
[noreply] Fixing patching error on missing dependencies (#17564)
[noreply] Merge pull request #17517 from [BEAM-14383] Improve "FailedRows" errors
[Heejong Lee] add test without mock
------------------------------------------
[...truncated 329.20 KB...]
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.22.9-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.26.0,>=1.25.9
Using cached botocore-1.25.9-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2769970 sha256=9cc9be49ba12636d6a0921516bc994bcbdb6a88e605ae2233ee2e01e4f136ec8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.24.0 azure-storage-blob-12.11.0 boto3-1.22.9 botocore-1.25.9 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fcf2d35f200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fcf2d35f9e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0507100427.1651938352.120670/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0507100427.1651938352.120670/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0507100427.1651938352.120670/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0507100427.1651938352.120670/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220507154552121646-4234'
createTime: '2022-05-07T15:45:53.140011Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-07_08_45_52-13998133551949395336'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0507100427'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-07T15:45:53.140011Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-07_08_45_52-13998133551949395336]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-07_08_45_52-13998133551949395336
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-07_08_45_52-13998133551949395336?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-07_08_45_52-13998133551949395336 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:54.953Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:55.704Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:55.813Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:55.864Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:55.902Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:55.965Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.052Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.090Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.122Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.157Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.186Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.224Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.257Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.284Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.309Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.470Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.532Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.579Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:45:56.601Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-07_08_45_52-13998133551949395336 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:46:19.012Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:46:27.154Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:46:51.387Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.452Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.553Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.586Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.673Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.730Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.798Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T15:53:05.885Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:00:41.823Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-07_08_45_52-13998133551949395336.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:00:41.852Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:00:41.960Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:00:42.101Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:00:42.124Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-07_08_45_52-13998133551949395336 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:01:16.423Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:01:16.458Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-07T20:01:16.485Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-07_08_45_52-13998133551949395336 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 16m 38s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ghq57sqyqphtc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #575
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/575/display/redirect?page=changes>
Changes:
[zyichi] Move master readme.md to 2.40.0
[noreply] [BEAM-14173] Fix Go Loadtests on Dataflow & partial fix for Flink
[noreply] Upgrade python sdk container requirements. (#17549)
[noreply] Merge pull request #17497: [BEAM-11205] Update GCP Libraries BOM version
[noreply] [BEAM-12603] Add retry on grpc data channel and remove retry from test.
[noreply] Merge pull request #17359: [BEAM-14303] Add a way to exclude output
[noreply] [BEAM-14347] Allow users to optimize DoFn execution with a single
[noreply] [BEAM-5878] Add (failing) kwonly-argument test (#17509)
------------------------------------------
[...truncated 329.76 KB...]
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.26.0,>=1.25.8
Using cached botocore-1.25.8-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2766904 sha256=04a6b17ed4a5982a4646e9d864711cb74f02adad69e619eef684e6fc68bc940a
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.8 botocore-1.25.8 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb08e6f4170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb08e6f4950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0506100420.1651852937.003077/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0506100420.1651852937.003077/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0506100420.1651852937.003077/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0506100420.1651852937.003077/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220506160217004048-5230'
createTime: '2022-05-06T16:02:18.292725Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-06_09_02_17-2928937170520540250'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0506100420'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-06T16:02:18.292725Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-06_09_02_17-2928937170520540250]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-06_09_02_17-2928937170520540250
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-06_09_02_17-2928937170520540250?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-06_09_02_17-2928937170520540250 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:21.154Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.658Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.733Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.769Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.800Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.846Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.874Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.906Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.941Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.966Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:22.988Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.011Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.057Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.096Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.126Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.160Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.320Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.384Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-06_09_02_17-2928937170520540250 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.428Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:23.480Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:46.469Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:02:50.771Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:03:18.592Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.291Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.403Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.445Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.560Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.616Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.693Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T16:09:58.764Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:00:39.344Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-06_09_02_17-2928937170520540250.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:00:39.377Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:00:39.579Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:00:39.720Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:00:39.766Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-06_09_02_17-2928937170520540250 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:01:14.273Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:01:14.315Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-06T20:01:14.343Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-06_09_02_17-2928937170520540250 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 18s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/jxaemmykxxshg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #574
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/574/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-9245] Unable to pull datatore Entity which contains dict
[bulat.safiullin] [BEAM-14382] [Website] add banner container for with css, images, html
[Jan Lukavský] [BEAM-14196] add test verifying output watermark propagation in bundle
[Jan Lukavský] [BEAM-14196] Fix FlinkRunner mid-bundle output watermark handling
[nielm] [BEAM-14405] Fix NPE when ProjectID is not specified in a template
[bulat.safiullin] [BEAM-14382] change mobile banner img, add padding to banner section
[ahmedabualsaud] fix test decotrator typo
[noreply] Merge pull request #17440 from [BEAM-14329] Enable exponential backoff
[noreply] [BEAM-11104] Fix output forwarding issue for ProcessContinuations
[noreply] re-add testing package to pydoc (#17524)
[Heejong Lee] add test
[noreply] [BEAM-14250] Amended the workaround (#17531)
[noreply] [BEAM-11104] Fix broken split result validation (#17546)
[noreply] Fixed a SQL and screenshots in the Beam SQL blog (#17545)
[noreply] Merge pull request #17417: [BEAM-14388] Address some performance
[noreply] [BEAM-14386] [Flink] Support for scala 2.12 (#17512)
[noreply] [BEAM-14294] Worker changes to support trivial Batched DoFns (#17384)
[zyichi] Moving to 2.40.0-SNAPSHOT on master branch.
[noreply] [BEAM-14048] [CdapIO] Add ConfigWrapper for building CDAP PluginConfigs
------------------------------------------
[...truncated 330.18 KB...]
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.40.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.26.0,>=1.25.7
Using cached botocore-1.25.7-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.40.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.40.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.40.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.40.0.dev0-py3-none-any.whl size=2766392 sha256=7118c32d29ab186ee10c2340b20c3af4f80ce921ab6befeff70864226f9d2194
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.40.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.7 botocore-1.25.7 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.40.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fea6f2d8170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fea6f2d8950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0505100417.1651766455.004017/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0505100417.1651766455.004017/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0505100417.1651766455.004017/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0505100417.1651766455.004017/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220505160055004971-2689'
createTime: '2022-05-05T16:00:56.643556Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-05_09_00_55-11769499013114091023'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0505100417'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-05T16:00:56.643556Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-05_09_00_55-11769499013114091023]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-05_09_00_55-11769499013114091023
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-05_09_00_55-11769499013114091023?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-05_09_00_55-11769499013114091023 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:51.827Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:52.775Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:52.823Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:52.873Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:52.910Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:52.965Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.008Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.040Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.109Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.177Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.209Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.281Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.316Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.359Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.397Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.617Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.708Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.772Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:01:53.815Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-05_09_00_55-11769499013114091023 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:02:15.664Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:02:24.663Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:02:48.700Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:00.775Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:00.920Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:00.955Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:01.035Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:01.109Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:01.252Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T16:09:01.346Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:00:42.754Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-05_09_00_55-11769499013114091023.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:00:42.794Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:00:42.961Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-05_09_00_55-11769499013114091023 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:00:43.085Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:00:43.104Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:01:22.449Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:01:22.503Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-05T20:01:22.535Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-05_09_00_55-11769499013114091023 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 1m 44s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/7bciwxkxg5cb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #573
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/573/display/redirect?page=changes>
Changes:
[noreply] fix: JDBC config schema fields order
[Brian Hulette] Revert "Merge pull request #17255 from kileys/test-revert"
[Brian Hulette] BEAM-14231: bypass schema cache for
[noreply] [BEAM-13657] Follow up update version warning in __init__ (#17493)
[noreply] Merge pull request #17431 from [BEAM-14273] Add integration tests for BQ
[noreply] Merge pull request #17205 from [BEAM-14145] [Website] add carousel to
[noreply] [BEAM-14064] fix es io windowing (#17112)
[noreply] [BEAM-13670] Upgraded ipython from v7 to v8 (#17529)
[noreply] [BEAM-11104] Enable ProcessContinuation return values, add unit test
[Robert Bradshaw] [BEAM-14403] Allow Prime to be used with legacy workers.
[noreply] [BEAM-11106] Support drain in Go SDK (#17432)
[noreply] add __Init__ to inference. (#17514)
------------------------------------------
[...truncated 328.90 KB...]
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.26.0,>=1.25.6
Using cached botocore-1.25.6-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.46.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2759580 sha256=4f146c474b1a5f3c9667d1a13ab9fea9f96ba05cfeea289e845034a796cab769
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.6 botocore-1.25.6 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.46.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2492833e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2492838680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651679455.910396/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651679455.910396/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651679455.910396/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651679455.910396/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220504155055911344-9566'
createTime: '2022-05-04T15:50:57.052430Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-04_08_50_56-13240567463436534960'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-04T15:50:57.052430Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-04_08_50_56-13240567463436534960]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-04_08_50_56-13240567463436534960
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-04_08_50_56-13240567463436534960?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-04_08_50_56-13240567463436534960 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.035Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.824Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.864Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.917Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.948Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:50:59.988Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.022Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.058Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.091Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.113Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.147Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.182Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.213Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.247Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.271Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.303Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.494Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.552Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.623Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:00.669Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-04_08_50_56-13240567463436534960 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:25.094Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:25.126Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:28.525Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:35.292Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:51:59.075Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:12.903Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:12.973Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:13.010Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:13.084Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:13.137Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:13.192Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T15:58:13.258Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:00:40.238Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-04_08_50_56-13240567463436534960.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:00:40.260Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:00:41.319Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:00:41.363Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:00:41.387Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-04_08_50_56-13240567463436534960 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:01:14.778Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:01:14.816Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-04T20:01:14.836Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-04_08_50_56-13240567463436534960 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 11m 44s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/kdodeimagdwru
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #572
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/572/display/redirect?page=changes>
Changes:
[yathu] [BEAM-14375] Fix Java Wordcount Dataflow postcommit
[Robert Bradshaw] Allow arithmetic between deferred scalars.
[noreply] [BEAM-14390] Set user-agent when pulling licenses to avoid 403s (#17521)
[noreply] [BEAM-8688] Upgrade GCSIO to 2.2.6 (#17486)
[noreply] [BEAM-14253] patch SubscriptionPartitionLoader to work around a dataflow
[noreply] Add website link log to notify user of pre-build workflow. (#17498)
[noreply] [BEAM-11105] Add timestamp observing watermark estimation (#17476)
[noreply] Merge pull request #17487 from Adding user-agent to GCS client in Python
[noreply] [BEAM-10265] Display error message if trying to infer recursive schema
[noreply] [BEAM-12575] Upgraded ipykernel from v5 to v6 (#17526)
[noreply] [BEAM-11105] Add docs + CHANGES.md entry for Go Watermark Estimation
[noreply] Merge pull request #17380 from [BEAM-14314][BEAM-9532] Add last_updated
------------------------------------------
[...truncated 329.11 KB...]
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.26.0,>=1.25.5
Using cached botocore-1.25.5-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2740861 sha256=4b1efe05012cf2f198db8f61396e39c1f0f14ce125edf2f356a4f8558575ef74
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.5 botocore-1.25.5 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd8db172dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd8db1735f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651591758.946113/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651591758.946113/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651591758.946113/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651591758.946113/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220503152918947038-5293'
createTime: '2022-05-03T15:29:19.926456Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-03_08_29_19-8254765706794469445'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-03T15:29:19.926456Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-03_08_29_19-8254765706794469445]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-03_08_29_19-8254765706794469445
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-03_08_29_19-8254765706794469445?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-03_08_29_19-8254765706794469445 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:23.349Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.172Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.215Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.245Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.274Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.305Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.344Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.382Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.420Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.494Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.549Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.579Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.624Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.656Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.698Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.736Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.879Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:24.936Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:25.004Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-03_08_29_19-8254765706794469445 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:25.036Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:48.084Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:29:55.927Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:30:20.712Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.495Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.577Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.613Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.674Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.735Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.796Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T15:36:05.860Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
Daemon will be stopped at the end of the build after the daemon was no longer found in the daemon registry
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:00:40.013Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-03_08_29_19-8254765706794469445.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:00:40.052Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:00:40.246Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:00:40.351Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:00:40.404Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-03_08_29_19-8254765706794469445 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:01:14.886Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:01:14.958Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-03T20:01:14.986Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-03_08_29_19-8254765706794469445 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1670, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 33m 8s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/eaun622rx5iye
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #571
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/571/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11105] Add manual watermark estimation (#17475)
------------------------------------------
[...truncated 328.48 KB...]
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.4
Using cached botocore-1.25.4-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2737178 sha256=0e968f15104525e987427d93dd9e53e0b5cc806372404ee426b4301448e2bf0f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.4 botocore-1.25.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc3c07abcb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc3c07ac4d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651506577.749512/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651506577.749512/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651506577.749512/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651506577.749512/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220502154937750454-6199'
createTime: '2022-05-02T15:49:38.841702Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-02_08_49_38-6566892061520285346'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-02T15:49:38.841702Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-02_08_49_38-6566892061520285346]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-02_08_49_38-6566892061520285346
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-02_08_49_38-6566892061520285346?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-02_08_49_38-6566892061520285346 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:45.469Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.481Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.543Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.576Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.603Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.639Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.668Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.689Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.711Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.734Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.757Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.787Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.811Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.844Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.879Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:46.903Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:47.043Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:47.103Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:47.204Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:49:47.262Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-02_08_49_38-6566892061520285346 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:50:09.333Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:50:18.976Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:50:42.190Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:55.971Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.046Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.070Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.142Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.187Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.245Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T15:56:56.315Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:00:30.268Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-02_08_49_38-6566892061520285346.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:00:30.296Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:00:30.452Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:00:30.549Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:00:30.583Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-02_08_49_38-6566892061520285346 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:01:02.662Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:01:02.702Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-02T20:01:02.728Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-02_08_49_38-6566892061520285346 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 12m 36s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/uwckbjskahiss
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #570
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/570/display/redirect?page=changes>
Changes:
[noreply] Revert "Improvement to Seed job configuration to launch against PRs
[ilion.beyst] Minor: fix typo
[noreply] Merge pull request #17422 from [BEAM-14344]: remove tracing from
------------------------------------------
[...truncated 328.31 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.26.0,>=1.25.4
Using cached botocore-1.25.4-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2737178 sha256=74d5dc2aec26d09c6a16f0aec68ba45128c29b2da0dc013fef5dbb510c70ae4f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.4 botocore-1.25.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0d222c1cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0d222c24d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651417735.049388/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651417735.049388/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651417735.049388/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651417735.049388/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220501150855050352-8891'
createTime: '2022-05-01T15:08:55.931841Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-05-01_08_08_55-15359025150078411460'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-05-01T15:08:55.931841Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-05-01_08_08_55-15359025150078411460]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-05-01_08_08_55-15359025150078411460
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-01_08_08_55-15359025150078411460?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-01_08_08_55-15359025150078411460 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:58.176Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.304Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.371Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.403Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.424Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.457Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.493Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.520Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.543Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.576Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.604Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.628Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.663Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.687Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.721Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.745Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.863Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.956Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:08:59.994Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:09:00.033Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-01_08_08_55-15359025150078411460 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:09:10.046Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:09:22.491Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:09:55.609Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.267Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.321Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.377Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.443Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.498Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.562Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T15:16:00.644Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:00:41.157Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-05-01_08_08_55-15359025150078411460.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:00:41.183Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:00:41.376Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:00:41.441Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:00:41.457Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-01_08_08_55-15359025150078411460 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:01:16Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:01:16.409Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-05-01T20:01:16.471Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-05-01_08_08_55-15359025150078411460 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 53m 32s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ccfzuz4oiz7x6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #569
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/569/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Add element weighting parameter to BatchElements.
[Robert Bradshaw] Clearer test.
[noreply] Revert "Merge pull request #17260 from [BEAM-13229] [Website] bug side
[noreply] [BEAM-14001] Add missing test cases to existing suites in exec package
[noreply] [BEAM-14243] Add staticcheck to Github Actions Precommits (#17479)
[noreply] [BEAM-14368][BEAM-13984]Change model loading from constructor to
[noreply] [BEAM-13983] changed file name from sklearn_loader to sklearn_inference
[noreply] Add SQL in Notebooks blog post (#17481)
[noreply] Merge pull request #17404: [BEAM-13990] support date and timestamp
------------------------------------------
[...truncated 327.87 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.26.0,>=1.25.4
Using cached botocore-1.25.4-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2737180 sha256=81208c181c57dffb000d972611cecb1f49e699976e8b15f120b1bbf62444ab82
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.4 botocore-1.25.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fad93031cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fad930324d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651333472.756675/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651333472.756675/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651333472.756675/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651333472.756675/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220430154432757581-5275'
createTime: '2022-04-30T15:44:33.715578Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-30_08_44_33-5989937202522807822'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-30T15:44:33.715578Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-30_08_44_33-5989937202522807822]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-30_08_44_33-5989937202522807822
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-30_08_44_33-5989937202522807822?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-30_08_44_33-5989937202522807822 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:36.107Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:36.928Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:36.977Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.017Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.052Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.190Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.289Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.458Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.553Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.663Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.692Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.727Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.748Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.776Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.799Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:37.964Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:38.029Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:38.077Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:44:38.108Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-30_08_44_33-5989937202522807822 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:45:00.194Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:45:03.161Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:45:33.354Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.016Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.092Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.128Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.236Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.296Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.362Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T15:51:56.426Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:00:42.906Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-30_08_44_33-5989937202522807822.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:00:42.961Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-30_08_44_33-5989937202522807822 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:00:43.882Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:00:44.185Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:00:44.222Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:01:22.083Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:01:22.124Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-30T20:01:22.144Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-30_08_44_33-5989937202522807822 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 18m 1s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/qd5nqzuprzikc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #568
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/568/display/redirect?page=changes>
Changes:
[ihr] Update Java katas to Beam 2.38
[noreply] [BEAM-14369] Fix "target/options: no such file or directory" error while
[noreply] [BEAM-14297] Enable nullable key and value arrays for xlang kafka io
[noreply] Merge pull request #17444 from [BEAM-14310] [Website] bug home
[noreply] Merge pull request #17388 from [BEAM-14311] [Website] Home Page
[noreply] [BEAM-14376] Typo in method description doc
[noreply] Add default classpath when not present (#17491)
[thiagotnunes] fix: update javadocs for ChangeStreamMetrics
[noreply] Merge pull request #17443 from [BEAM-12164]: use the end timestamp for
[noreply] Merge pull request #17260 from [BEAM-13229] [Website] bug side nav
[noreply] [BEAM-14351] Fix the template and move the announcement to the next
------------------------------------------
[...truncated 328.91 KB...]
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.3
Using cached botocore-1.25.3-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736894 sha256=1ee72e43fc3368e447f9fb5271518b7be87d37e2f1c9cb26c3f8fbf3cc2983fe
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.3 botocore-1.25.3 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f05338669e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0533867200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651245988.746536/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651245988.746536/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651245988.746536/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651245988.746536/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220429152628747471-7084'
createTime: '2022-04-29T15:26:29.744312Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-29_08_26_29-12744793770802617624'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-29T15:26:29.744312Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-29_08_26_29-12744793770802617624]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-29_08_26_29-12744793770802617624
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-29_08_26_29-12744793770802617624?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-29_08_26_29-12744793770802617624 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:32.892Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.781Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.834Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.859Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.889Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.928Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.952Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.972Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:33.995Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.053Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.074Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.102Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.134Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.164Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.190Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.408Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.475Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.513Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:34.535Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-29_08_26_29-12744793770802617624 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:56.699Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:26:56.727Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:27:06.961Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:27:08.370Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:27:30.111Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.341Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.482Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.514Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.565Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.622Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.686Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T15:33:40.765Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:00:43.150Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-29_08_26_29-12744793770802617624.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:00:43.210Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:00:43.387Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:00:43.581Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:00:43.610Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-29_08_26_29-12744793770802617624 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:01:19.048Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:01:19.121Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-29T20:01:19.158Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-29_08_26_29-12744793770802617624 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 36m 2s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ugaygtycjokki
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #567
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/567/display/redirect?page=changes>
Changes:
[nick.caballero] [BEAM-14363] Fixes WatermarkParameters builder for Kinesis
[noreply] [BEAM-11104] Add Checkpointing split to Go SDK (#17386)
[noreply] Merge pull request #17226 from [BEAM-14204] [Playground] Tests for
[noreply] [BEAM-13015, BEAM-14184] Address unbounded number of messages being
[noreply] Improvement to Seed job configuration to launch against PRs (#17468)
[noreply] [BEAM-13983] Small changes to sklearn runinference (#17459)
[chamikaramj] Renames ExternalPythonTransform to PythonExternalTransform
[noreply] [BEAM-14351] Inherit from Coder. (#17437)
------------------------------------------
[...truncated 328.29 KB...]
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.2
Using cached botocore-1.25.2-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736897 sha256=c3deb0c0fe1ce5c64bf26d4caf19ee05cae6cfa10a1e695af24414c9d3991943
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.2 botocore-1.25.2 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8ce0ab79e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8ce0ab6200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651160822.668724/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651160822.668724/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651160822.668724/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0428111049.1651160822.668724/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220428154702669644-8640'
createTime: '2022-04-28T15:47:06.328574Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-28_08_47_03-9881401568097228350'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0428111049'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-28T15:47:06.328574Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-28_08_47_03-9881401568097228350]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-28_08_47_03-9881401568097228350
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-28_08_47_03-9881401568097228350?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-28_08_47_03-9881401568097228350 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:11.555Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.403Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.459Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.489Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.519Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.569Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.590Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.629Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.652Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.677Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.701Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.728Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.754Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.786Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.810Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:12.831Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:13.010Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:13.130Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:13.201Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:13.232Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-28_08_47_03-9881401568097228350 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:38.961Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:38.996Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:46.959Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:47:49.239Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:48:12.529Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:16.870Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:16.947Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:16.973Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:17.028Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:17.108Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:17.160Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T15:54:17.224Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:00:39.438Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-28_08_47_03-9881401568097228350.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:00:39.472Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:00:39.652Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:00:39.754Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:00:39.803Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-28_08_47_03-9881401568097228350 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:01:16.500Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:01:16.562Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-28T20:01:16.598Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-28_08_47_03-9881401568097228350 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 15m 28s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/mnf3iqijobtay
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #566
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/566/display/redirect?page=changes>
Changes:
[msbukal] FhirIO: use .search() or .searchType instead of .setResourceType()
[noreply] Remove unnecessary decorator from RunInference interface (#17463)
[noreply] [BEAM-13590] Minor deprecated warning fix (#17453)
[noreply] [BEAM-12164]: fix the negative throughput issue (#17461)
[noreply] Updated goldens for the screen diff integration tests (#17467)
[noreply] fixes copy by value error for bytes.Buffer in Error (#17469)
[noreply] Merge pull request #17354 from [BEAM-14170] - Create a test that runs
[noreply] Merge pull request #17447 from [BEAM-14357] Fix
[noreply] [BEAM-14324, BEAM-14325] Staticcheck cleanup in test files (#17393)
[noreply] BEAM-14187 Fix NPE (#17454)
[noreply] [BEAM-11105] Stateful watermark estimation (#17374)
[noreply] [BEAM-14304] implement parquetio to read/write parquet files (#17347)
------------------------------------------
[...truncated 328.56 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.26.0,>=1.25.1
Using cached botocore-1.25.1-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736891 sha256=2f39ce750b994b07e0d093be45b6949460978391959afdfa7ecd500ef3112db0
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.1 botocore-1.25.1 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.0 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.11 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.36 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe03c4898c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe03c4880e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0427100426.1651074624.330787/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0427100426.1651074624.330787/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0427100426.1651074624.330787/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0427100426.1651074624.330787/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220427155024331753-6948'
createTime: '2022-04-27T15:50:26.391601Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-27_08_50_24-17485048539983292671'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0427100426'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-27T15:50:26.391601Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-27_08_50_24-17485048539983292671]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-27_08_50_24-17485048539983292671
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-27_08_50_24-17485048539983292671?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-27_08_50_24-17485048539983292671 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:28.541Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.727Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.775Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.834Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.863Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.897Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.927Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.966Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:33.995Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.020Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.049Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.079Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.116Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.145Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.174Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.202Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.359Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.449Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.489Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:34.523Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-27_08_50_24-17485048539983292671 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:50:56.926Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:51:01.707Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:51:28.960Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.044Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.107Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.143Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.211Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.272Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.349Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T15:58:00.418Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:00:38.849Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-27_08_50_24-17485048539983292671.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:00:38.877Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:00:39.047Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:00:39.201Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:00:39.294Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-27_08_50_24-17485048539983292671 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:01:20.910Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:01:20.943Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-27T20:01:20.964Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-27_08_50_24-17485048539983292671 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 12m 32s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/qiez6sauhl7l4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #565
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/565/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14343] Allow expansion service override in ExternalPythonTransform
[Heejong Lee] update
[Heejong Lee] allows remote host
[Heejong Lee] improve compatibility with python rowcoder
[ahmedabualsaud] added tempLocation to test pipeline options
[ahmedabualsaud] using tempRoot for temp bucket location
[ahmedabualsaud] small fixes
[noreply] [BEAM-14320] Update programming-guide w/Java GroupByKey example (#17369)
[noreply] Minor: Fix release script for `current` symlinks (#17457)
[noreply] Minor: fix typo (#17452)
[noreply] Change return type for PytorchInferenceRunner (#17460)
[noreply] [BEAM-13608] JmsIO dynamic topics feature (#17163)
[Heejong Lee] add test
------------------------------------------
[...truncated 329.02 KB...]
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Downloading cryptography-37.0.0-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/4.0 MB 43.0 MB/s eta 0:00:00
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.26.0,>=1.25.0
Using cached botocore-1.25.0-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736964 sha256=d087cc041bf66445e3789144597d8ee7d85452ac6b260683ba05c4fd08d11073
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.22.0 botocore-1.25.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-37.0.0 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb83c2038c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb83c2020e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0426100422.1650988539.548684/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0426100422.1650988539.548684/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0426100422.1650988539.548684/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0426100422.1650988539.548684/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220426155539549609-7974'
createTime: '2022-04-26T15:55:41.522179Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-26_08_55_40-9466384311318386118'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0426100422'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-26T15:55:41.522179Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-26_08_55_40-9466384311318386118]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-26_08_55_40-9466384311318386118
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-26_08_55_40-9466384311318386118?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-26_08_55_40-9466384311318386118 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:43.235Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.049Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.082Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.109Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.142Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.174Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.201Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.227Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.249Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.276Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.309Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.341Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.369Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.424Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.488Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.579Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.832Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.880Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.918Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:55:44.948Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-26_08_55_40-9466384311318386118 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:56:14.720Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:56:17.360Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T15:56:42.793Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:18.808Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:18.875Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:18.898Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:18.962Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:19.036Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:19.107Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T16:02:19.184Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:00:39.427Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-26_08_55_40-9466384311318386118.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:00:39.454Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:00:39.645Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:00:39.696Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:00:39.740Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-26_08_55_40-9466384311318386118 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:01:12.290Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:01:12.333Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-26T20:01:12.356Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-26_08_55_40-9466384311318386118 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 49s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ya45w2askymns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/564/display/redirect>
Changes:
------------------------------------------
[...truncated 328.66 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.46
Using cached botocore-1.24.46-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736970 sha256=3c4b699ed695d57e034ab22f4419a14c66c3637d390a9e7f17e2c8daacf7d243
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.46 botocore-1.24.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa0ff369e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa0ff36a680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0425100404.1650901668.237874/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0425100404.1650901668.237874/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0425100404.1650901668.237874/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0425100404.1650901668.237874/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220425154748238798-1199'
createTime: '2022-04-25T15:47:49.203597Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-25_08_47_48-9161074525217462149'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0425100404'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-25T15:47:49.203597Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-25_08_47_48-9161074525217462149]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-25_08_47_48-9161074525217462149
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-25_08_47_48-9161074525217462149?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-25_08_47_48-9161074525217462149 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:51.633Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:52.870Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:52.900Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:52.933Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:52.989Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.019Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.186Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.348Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.404Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.438Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.462Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.496Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.529Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.559Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.592Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.652Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.847Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.917Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:53.964Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:47:54Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-25_08_47_48-9161074525217462149 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:48:18.571Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:48:22.092Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:48:53.099Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:54:59.766Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:54:59.831Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:54:59.857Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:54:59.914Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:54:59.969Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:55:00.040Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T15:55:00.100Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:00:42.408Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-25_08_47_48-9161074525217462149.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:00:42.443Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:00:43.433Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:00:43.512Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:00:43.531Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-25_08_47_48-9161074525217462149 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:01:14.427Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:01:14.470Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-25T20:01:14.495Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-25_08_47_48-9161074525217462149 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 36s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/56f4f5iadnxlc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/563/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13953] added documentation for BQ Storage Write API (#17391)
------------------------------------------
[...truncated 327.92 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.46
Using cached botocore-1.24.46-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736970 sha256=8b674f6eed2cce8d9a2fd35499ebfb786eaf23fb59cfb559ab26feff52a204f5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.46 botocore-1.24.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9c47232e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9c47233680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0424100405.1650813883.978827/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0424100405.1650813883.978827/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0424100405.1650813883.978827/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0424100405.1650813883.978827/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220424152443979754-6743'
createTime: '2022-04-24T15:24:45.079902Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-24_08_24_44-15267330162793250927'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0424100405'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-24T15:24:45.079902Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-24_08_24_44-15267330162793250927]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-24_08_24_44-15267330162793250927
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-24_08_24_44-15267330162793250927?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-24_08_24_44-15267330162793250927 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:46.999Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:47.873Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:47.904Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:47.939Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:47.972Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.006Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.023Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.056Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.105Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.135Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.172Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.204Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.237Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.291Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.319Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.355Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.522Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.575Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.665Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:24:48.713Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-24_08_24_44-15267330162793250927 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:25:11.061Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:25:22.742Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:25:43.510Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.222Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.296Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.322Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.409Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.462Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.527Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T15:31:41.584Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:00:37.294Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-24_08_24_44-15267330162793250927.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:00:37.333Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:00:37.465Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:00:37.508Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:00:37.560Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-24_08_24_44-15267330162793250927 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:01:12.123Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:01:12.160Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-24T20:01:12.187Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-24_08_24_44-15267330162793250927 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 37m 39s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/gm6sxhku6jz2w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/562/display/redirect?page=changes>
Changes:
[Andrew Pilloud] [BEAM-14321] SQL passes Null for Null aggregates
[noreply] Create apache-hop-with-dataflow.md
[noreply] Add files via upload
[noreply] Delete website/www/site/content/en/blog/apache-hop-with-dataflow
[noreply] Add files via upload
[Andrew Pilloud] [BEAM-14348] Upgrade to ZetaSQL 2022.04.1
[Andrew Pilloud] [BEAM-13735] Enable ZetaSQL tests for Java 17
[noreply] Update apache-hop-with-dataflow.md
[noreply] Update apache-hop-with-dataflow.md
[noreply] Update apache-hop-with-dataflow.md
[danielamartinmtz] Moved up get-credentials instruction for getting the kubeconfig file
[noreply] [BEAM-13657] Add python 3.6 update to CHANGES.md (#17435)
[noreply] Merge pull request #17438: [BEAM-8127] The GCP module to declare
[noreply] Merge pull request #17428: [BEAM-14326] Make sure BigQuery daemon thread
[noreply] [BEAM-14301] Add lint:ignore to noescape() func (#17355)
[noreply] [BEAM-14286] Remove unused vars in harness package (#17392)
[noreply] [BEAM-14327] Convert Results to QueryResults directly (#17398)
[noreply] [BEAM-14302] Simplify boolean check in fn.go (#17399)
[noreply] [BEAM-13983] Sklearn Loader for RunInference (#17368)
[noreply] Update authors.yml
[noreply] [BEAM-14358] add retry to connect to testcontainer (#17449)
[noreply] [BEAM-13106] Bump flink docs to 1.14 (#17430)
------------------------------------------
[...truncated 328.27 KB...]
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.46
Using cached botocore-1.24.46-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736970 sha256=866218795f199d2846a2e7dddcf544d40a1ab7671585e5d9c3dcebcb8578bad5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.46 botocore-1.24.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 joblib-1.1.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 threadpoolctl-3.1.0 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fec79514e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fec79515680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0423100420.1650728124.648628/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0423100420.1650728124.648628/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0423100420.1650728124.648628/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0423100420.1650728124.648628/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220423153524649534-2130'
createTime: '2022-04-23T15:35:26.490356Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-23_08_35_25-5646180315786722022'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0423100420'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-23T15:35:26.490356Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-23_08_35_25-5646180315786722022]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-23_08_35_25-5646180315786722022
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-23_08_35_25-5646180315786722022?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-23_08_35_25-5646180315786722022 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:28.586Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:29.874Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:29.939Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:29.973Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.009Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.074Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.109Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.181Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.210Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.248Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.280Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.304Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.336Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.368Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.404Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.539Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.594Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.643Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:30.677Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-23_08_35_25-5646180315786722022 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:51.990Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:35:52.721Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:36:26.316Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:10.972Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.053Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.088Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.155Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.218Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.280Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T15:42:11.345Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:00:41.694Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-23_08_35_25-5646180315786722022.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:00:41.719Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:00:42.029Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:00:42.107Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:00:42.126Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-23_08_35_25-5646180315786722022 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:01:15.314Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:01:15.356Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-23T20:01:15.385Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-23_08_35_25-5646180315786722022 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 27m 2s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/crb6mhuux66te
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/561/display/redirect?page=changes>
Changes:
[mmack] [BEAM-14335] Spotless Spark sources
[mmack] [BEAM-14345] Force paranamer 2.8 for Spark Hadoop version tests to avoid
[noreply] BEAM-13582 Fixing broken links in the documentation (#17300)
[kamil.bregula] Revert "[BEAM-14300] Fix Java precommit failure"
[kamil.bregula] Revert "Merge pull request #17223 from [BEAM-14215] Improve argument
[noreply] [BEAM-13657] Sunset python 3.6 (#17252)
[noreply] Removes unsupported Python 3.6 from the release validation script
[noreply] [BEAM-13984] Implement RunInference for PyTorch (#17196)
[noreply] [BEAM-13945] add json type support for java bigquery connector (#17209)
[noreply] [BEAM-14346] Fix incorrect error case index in ret2() (#17425)
[noreply] [BEAM-14342] Fix wrong default buffer type in fn_runner (#17420)
[noreply] Updates opencensus-api dependency to the latest version - 0.31.0
[noreply] [BEAM-14306] Add unit testing to pane coder (#17370)
[noreply] Updated the dep and golden for screen diff integration tests (#17442)
------------------------------------------
[...truncated 323.72 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.45
Using cached botocore-1.24.45-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736945 sha256=d8d6cf89956d14972103b927ea26839ed81c0abafb5e6fe6529f2e21aadb4ae6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.45 botocore-1.24.45 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fcfc6574e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fcfc6573680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0422100426.1650642490.436078/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0422100426.1650642490.436078/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0422100426.1650642490.436078/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0422100426.1650642490.436078/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220422154810436993-2070'
createTime: '2022-04-22T15:48:11.512906Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-22_08_48_10-9162201865597748891'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0422100426'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-22T15:48:11.512906Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-22_08_48_10-9162201865597748891]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-22_08_48_10-9162201865597748891
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-22_08_48_10-9162201865597748891?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-22_08_48_10-9162201865597748891 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:14.021Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.187Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.228Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.268Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.304Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.345Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.472Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.506Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.547Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.580Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.604Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.627Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.664Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.692Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.738Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.766Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.898Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:15.978Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:16.031Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:16.061Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-22_08_48_10-9162201865597748891 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:36.194Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:41.206Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:41.239Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:48:51.403Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:49:14.241Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.160Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.378Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.453Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.560Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.632Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.701Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T15:55:22.761Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:00:37.255Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-22_08_48_10-9162201865597748891.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:00:37.627Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-22_08_48_10-9162201865597748891 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:00:38.711Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:00:38.942Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:00:38.970Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:01:15.245Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:01:15.358Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-22T20:01:15.426Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 18s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/liousr3lml222
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/560/display/redirect?page=changes>
Changes:
[vachan] Annotating Read API tests.
[bulat.safiullin] [BEAM-14247] [Website] add image
[bulat.safiullin] [BEAM-14247] [Website] center image
[mattcasters] BEAM-1857 : CHANGES.md entry for 2.38.0
[mmack] [BEAM-14323] Improve IDE integration of Spark cross version builds
[noreply] [BEAM-14112] Fixed ReadFromBigQuery with Interactive Beam (#17306)
[noreply] Update .asf.yaml (#17409)
[noreply] [BEAM-14336] Sickbay flight delays test - dataset seems to be missing
[noreply] [BEAM-14338] Update watermark unit tests to use time.Time.Equals()
[noreply] [BEAM-14328] Tweaks to "Differences from pandas" page (#17413)
[Andrew Pilloud] [BEAM-14253] Disable broken test pending Dataflow fix
[yiru] fix: BigQuery Storage Connector trace id population missing bracket
[noreply] [BEAM-14330] Temporarily disable the clusters auto-cleanup (#17400)
[noreply] Update Beam website to release 2.38.0 (#17378)
[noreply] [BEAM-14213] Add API and construction time validation for Batched DoFns
[noreply] Minor: Update release guide regarding archive.apache.org (#17419)
[noreply] [BEAM-14017] beam_PreCommit_CommunityMetrics_Cron test failing (#17396)
------------------------------------------
[...truncated 323.77 KB...]
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.44-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.44
Using cached botocore-1.24.44-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2736888 sha256=77b1fcc7493b5c5ed3c371bb63f062a59b713502cda7dd2f46168704f2ce01da
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.44 botocore-1.24.44 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3e96410e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3e96412680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0421100441.1650556262.386747/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0421100441.1650556262.386747/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0421100441.1650556262.386747/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0421100441.1650556262.386747/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220421155102387692-6505'
createTime: '2022-04-21T15:51:04.633883Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-21_08_51_02-1239703817594666878'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0421100441'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-21T15:51:04.633883Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-21_08_51_02-1239703817594666878]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-21_08_51_02-1239703817594666878
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-21_08_51_02-1239703817594666878?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-21_08_51_02-1239703817594666878 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:18.440Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:19.981Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.233Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.278Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.333Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.396Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.450Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.490Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.540Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.593Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.631Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.664Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.719Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.777Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.813Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:20.849Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:21.071Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:21.187Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:21.239Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:21.281Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-21_08_51_02-1239703817594666878 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:43.442Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:51:48.317Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:52:17.813Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.039Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.109Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.134Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.251Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.298Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.370Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T15:59:00.424Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:00:37.857Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-21_08_51_02-1239703817594666878.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:00:37.887Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:00:38.107Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:00:38.214Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:00:38.245Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-21_08_51_02-1239703817594666878 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:01:10.877Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:01:10.932Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-21T20:01:10.960Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-21_08_51_02-1239703817594666878 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1661, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 11m 21s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/5rwmq5bhysc5u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/559/display/redirect?page=changes>
Changes:
[andyye333] Change func to PTransform
[noreply] Populate actual dataflow job id to bigquery write trace id (#17130)
[relax] mark static thread as a daemon thread
[noreply] [BEAM-13866] Add miscellaneous exec unit tests (#17363)
------------------------------------------
[...truncated 325.41 KB...]
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.35-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.43
Using cached botocore-1.24.43-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2730159 sha256=31debbc2e6d89110d034ce099e7bafd353453e0211f220a5443cba6b87a87b5e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.43 botocore-1.24.43 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f17a0b19b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f17a0b1d3b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0420100408.1650469695.192232/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0420100408.1650469695.192232/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0420100408.1650469695.192232/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0420100408.1650469695.192232/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220420154815193217-4656'
createTime: '2022-04-20T15:48:16.681Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-20_08_48_15-12055478762876212656'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0420100408'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-20T15:48:16.681Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-20_08_48_15-12055478762876212656]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-20_08_48_15-12055478762876212656
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-20_08_48_15-12055478762876212656?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-20_08_48_15-12055478762876212656 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:21.071Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:27.790Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:27.889Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:27.938Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.012Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.068Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.111Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.136Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.166Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.200Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.237Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.270Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.299Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.337Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.373Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.395Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.557Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.687Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.785Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:28.851Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-20_08_48_15-12055478762876212656 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:48:58.154Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:49:10.031Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:49:10.064Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:49:20.451Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:49:34.090Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:39.647Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:39.786Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:39.812Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:39.879Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:39.950Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:40.021Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T15:55:40.119Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:00:37.377Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-20_08_48_15-12055478762876212656.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:00:37.462Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:00:37.666Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:00:37.765Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:00:37.792Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-20_08_48_15-12055478762876212656 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:01:29.872Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:01:29.945Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-20T20:01:29.996Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 39s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/p6uegghha6zw6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/558/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14251] add output_coder_override to ExpansionRequest
[Heejong Lee] remove null
[rarokni] [BEAM-14307] Fix Slow Side input pattern bug in sample
[Heejong Lee] better error msg
[Heejong Lee] update from comments
[noreply] [BEAM-14316] Introducing KafkaIO.Read implementation compatibility
[noreply] [BEAM-14290] Address staticcheck warnings in the reflectx package
[noreply] [BEAM-14302] Simply bools in fn.go, genx_test.go (#17356)
[noreply] Merge pull request #17382: [BEAM-12356] Close DatasetService leak as
------------------------------------------
[...truncated 323.92 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.42
Using cached botocore-1.24.42-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2730159 sha256=2d165a95701506b4042ea31e153ff94c065378c4b19cd5d767226285e4a514fb
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.42 botocore-1.24.42 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f31576b2b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f31576b63b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0419100419.1650383274.658482/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0419100419.1650383274.658482/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0419100419.1650383274.658482/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0419100419.1650383274.658482/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220419154754659426-1439'
createTime: '2022-04-19T15:47:55.665853Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-19_08_47_55-12578305112851327411'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0419100419'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-19T15:47:55.665853Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-19_08_47_55-12578305112851327411]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-19_08_47_55-12578305112851327411
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-19_08_47_55-12578305112851327411?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-19_08_47_55-12578305112851327411 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:57.970Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:58.848Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:58.887Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:58.924Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:58.976Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.008Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.039Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.072Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.103Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.134Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.178Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.209Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.261Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.288Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.322Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.356Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.503Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.569Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.644Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:47:59.668Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-19_08_47_55-12578305112851327411 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:48:22.094Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:48:39.822Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:49:03.429Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.509Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.571Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.593Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.651Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.721Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.797Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T15:54:18.856Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:00:32.342Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-19_08_47_55-12578305112851327411.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:00:32.367Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:00:32.509Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:00:32.615Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:00:32.634Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-19_08_47_55-12578305112851327411 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:01:26.755Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:01:26.804Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-19T20:01:26.826Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 35s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/o42tpxzzxnkxe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/557/display/redirect>
Changes:
------------------------------------------
[...truncated 324.74 KB...]
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.42
Using cached botocore-1.24.42-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2729180 sha256=a63016bb0827e75a9b2cdfdf3092a1be36b2fb8c300ac7c73a8c06d6c6433cb9
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.42 botocore-1.24.42 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.2.0 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2ec5df5950> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2ec5df4170> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0418100409.1650296135.730276/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0418100409.1650296135.730276/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0418100409.1650296135.730276/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0418100409.1650296135.730276/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220418153535731192-3838'
createTime: '2022-04-18T15:35:36.762337Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-18_08_35_36-11666559735054610236'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0418100409'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-18T15:35:36.762337Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-18_08_35_36-11666559735054610236]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-18_08_35_36-11666559735054610236
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-18_08_35_36-11666559735054610236?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-18_08_35_36-11666559735054610236 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:40.696Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.698Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.732Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.760Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.793Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.824Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.853Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.887Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.918Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.961Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:46.993Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.018Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.078Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.129Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.166Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.207Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.351Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.404Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.445Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:35:47.480Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-18_08_35_36-11666559735054610236 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:36:15.912Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:36:27.999Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:36:50.752Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.199Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.272Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.296Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.364Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.409Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.456Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T15:41:57.515Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:00:40.247Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-18_08_35_36-11666559735054610236.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:00:40.284Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:00:40.571Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:00:40.712Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:00:40.735Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-18_08_35_36-11666559735054610236 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:01:33.714Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:01:33.791Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-18T20:01:33.822Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 26m 59s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/rqrr3vbddyxb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/556/display/redirect>
Changes:
------------------------------------------
[...truncated 325.47 KB...]
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.42
Using cached botocore-1.24.42-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2729180 sha256=b96e86df3e0a828b9d6439fb07264d427ffe8958b8e565037867cd300d2d80c0
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.42 botocore-1.24.42 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f121c80a830> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f121c80b050> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0417100416.1650209316.016704/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0417100416.1650209316.016704/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0417100416.1650209316.016704/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0417100416.1650209316.016704/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220417152836017650-3999'
createTime: '2022-04-17T15:28:36.993464Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-17_08_28_36-4309054532595874977'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0417100416'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-17T15:28:36.993464Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-17_08_28_36-4309054532595874977]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-17_08_28_36-4309054532595874977
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-17_08_28_36-4309054532595874977?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-17_08_28_36-4309054532595874977 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:39.426Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.361Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.394Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.420Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.446Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.483Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.515Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.551Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.574Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.612Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.646Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.681Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.717Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.748Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.784Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.814Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:40.957Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:41.014Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:41.059Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:41.083Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-17_08_28_36-4309054532595874977 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:28:53.327Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:29:20.724Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:29:43.095Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.029Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.100Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.124Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.190Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.246Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.311Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T15:35:15.371Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:00:30.731Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-17_08_28_36-4309054532595874977.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:00:30.773Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:00:30.920Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:00:31.432Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:00:31.471Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-17_08_28_36-4309054532595874977 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:01:24.636Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:01:24.701Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-17T20:01:24.736Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 33m 44s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/rerwcrdyaoy2i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/555/display/redirect?page=changes>
Changes:
[pandiana] BigQueryServicesImpl: reduce number of threads spawned by
[noreply] [BEAM-13204] Fix website bug where code tabs do not appear if the
------------------------------------------
[...truncated 326.82 KB...]
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.42-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.42
Using cached botocore-1.24.42-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2729180 sha256=c76a2c7d9433e5e66bb670c4ad78e6524217cfafd451e65eebca555d143de1f9
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.42 botocore-1.24.42 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7191b3a830> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7191b3e050> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0416100411.1650122609.833244/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0416100411.1650122609.833244/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0416100411.1650122609.833244/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0416100411.1650122609.833244/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220416152329834201-3418'
createTime: '2022-04-16T15:23:30.933225Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-16_08_23_30-8767902109433433862'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0416100411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-16T15:23:30.933225Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-16_08_23_30-8767902109433433862]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-16_08_23_30-8767902109433433862
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-16_08_23_30-8767902109433433862?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-16_08_23_30-8767902109433433862 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:35.716Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.546Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.595Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.631Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.664Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.704Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.734Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.767Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.813Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.845Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.878Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.913Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.948Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:38.982Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.014Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.048Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.150Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.205Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.254Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:39.277Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-16_08_23_30-8767902109433433862 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:23:53.341Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:24:19.746Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:24:43.007Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.345Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.395Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.427Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.494Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.548Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.624Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T15:30:31.671Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:00:30.295Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-16_08_23_30-8767902109433433862.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:00:30.317Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:00:30.506Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:00:30.588Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:00:30.604Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-16_08_23_30-8767902109433433862 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:01:20.870Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:01:20.910Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-16T20:01:20.933Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 38m 50s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/rnorgzk32bm4q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/554/display/redirect?page=changes>
Changes:
[Kenneth Knowles] Upgrade to Gradle 7.4
[Kenneth Knowles] Remove Python module dependency on Dataflow worker
[noreply] [BEAM-13925] Dont double assign committers if author or other reviewer
[noreply] [BEAM-13739] Remove deprecated shallow clone funcs (#17362)
[noreply] [BEAM-11104] Pipe Continuation to DataSource level (#17334)
[noreply] [BEAM-11105] Basic Watermark Estimation (Wall Clock Observing) (#17267)
[noreply] Respect output coder for TextIO. (#17367)
[noreply] Merge pull request #17200 from [BEAM-12164]: fix the autoscaling backlog
[noreply] [BEAM-17035] Call python3 directly when it is available. (#17366)
[noreply] Merge pull request #17375: [BEAM-8691] Declare newer
------------------------------------------
[...truncated 324.93 KB...]
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.41-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.41
Using cached botocore-1.24.41-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2729180 sha256=141f13b0e88276faae6145ab9d4aa5037fa7dba91540a445b51383948b93174d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.41 botocore-1.24.41 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f30c24007a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f30c2400f80> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0415100402.1650036592.423077/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0415100402.1650036592.423077/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0415100402.1650036592.423077/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0415100402.1650036592.423077/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220415152952423992-9860'
createTime: '2022-04-15T15:29:53.377589Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-15_08_29_52-7536532170273821471'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0415100402'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-15T15:29:53.377589Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-15_08_29_52-7536532170273821471]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-15_08_29_52-7536532170273821471
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-15_08_29_52-7536532170273821471?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-15_08_29_52-7536532170273821471 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:02.544Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.394Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.433Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.453Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.486Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.555Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.575Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.603Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.624Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.652Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.676Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.699Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.724Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.746Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.771Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.789Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.928Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:03.987Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:04.060Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:04.083Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-15_08_29_52-7536532170273821471 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:33.811Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 1 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:33.833Z: JOB_MESSAGE_DETAILED: Resized **** pool to 1, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:40.408Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:30:44.114Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:31:07.639Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:24.923Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:24.984Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:25.016Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:25.083Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:25.158Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:25.236Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T15:36:25.301Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:00:30.207Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-15_08_29_52-7536532170273821471.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:00:30.230Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:00:30.663Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:00:30.710Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:00:30.760Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-15_08_29_52-7536532170273821471 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:01:20.865Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:01:20.902Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-15T20:01:20.932Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 32m 28s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/bs3rgulerir7m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/553/display/redirect?page=changes>
Changes:
[relax] handle changing schemas in Storage API sink
[noreply] Fix a couple style issues (#17361)
[noreply] [BEAM-14287] Clean up staticcheck warnings in graph/coder (#17337)
[noreply] Improvements to dataflow job service for non-Python jobs. (#17338)
[noreply] Bump minimist (#17290)
[noreply] Bump ansi-regex (#17291)
[noreply] Bump nanoid (#17292)
[noreply] Bump lodash (#17293)
[noreply] Bump url-parse (#17294)
[noreply] Bump moment (#17328)
[noreply] Merge pull request #15549 from [BEAM-11997] Changed RedisIO
------------------------------------------
[...truncated 324.97 KB...]
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.35-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.40-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.40
Using cached botocore-1.24.40-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2729074 sha256=0257098da2bf4fa46370a1da21e6e7434faf6d5364eff917815b9aaa5fd5bb90
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.40 botocore-1.24.40 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5e2e0ce560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5e2e0ced40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0414100423.1649950687.316941/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0414100423.1649950687.316941/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0414100423.1649950687.316941/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0414100423.1649950687.316941/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220414153807317854-3556'
createTime: '2022-04-14T15:38:08.247926Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-14_08_38_07-845475702229582832'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0414100423'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-14T15:38:08.247926Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-14_08_38_07-845475702229582832]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-14_08_38_07-845475702229582832
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-14_08_38_07-845475702229582832?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-14_08_38_07-845475702229582832 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:10.638Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.136Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.175Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.204Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.236Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.278Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.318Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.340Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.568Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.679Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:12.796Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:13.046Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:13.228Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:13.335Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:13.467Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:14.630Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:15.175Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:15.395Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:15.512Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-14_08_38_07-845475702229582832 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:30.315Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:45.384Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:45.406Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:38:55.695Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:39:15.978Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.470Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.525Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.554Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.634Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.689Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.748Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T15:44:11.805Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T20:00:36.527Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-14_08_38_07-845475702229582832.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T20:00:36.557Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T20:00:36.663Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T20:00:36.792Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-14T20:00:36.818Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-14_08_38_07-845475702229582832 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1651, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 24m 24s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/3vn4ubcl4r43u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/552/display/redirect?page=changes>
Changes:
[Valentyn Tymofieiev] Add remaining Dataflow test suites for Python 3.9.
[Heejong Lee] [BEAM-14232] Only resolve artifacts in expanded environments for Java
[noreply] Fix test ordering issue (#17350)
[buqian] Do not pass null to MoreObjects.firstNonNull as default value
[ningkang0957] [BEAM-14288] Fixed flaky test
[noreply] [BEAM-14277] Disables Spanner change streams tests (#17346)
[noreply] [BEAM-14219] Run cleanup script to remove stale prebuilt SDK container
[Heejong Lee] [BEAM-14300] Fix Java precommit failure
[noreply] [BEAM-14116] Rollback "Chunk commit requests dynamically (#17004)"
[noreply] [BEAM-13982] A base class for run inference (#16970)
[ningkang0957] Enumerates all possible expected strings when asserting
[noreply] [BEAM-13966] Add pivot(), a non-deferred column operation on categorical
------------------------------------------
[...truncated 324.30 KB...]
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.35-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.39
Using cached botocore-1.24.39-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2728896 sha256=b5d35e00889ed7ef4240245d5969127ccb22f0871f3667e1e61fff4d680d41d1
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.39 botocore-1.24.39 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc7aa3a14d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc7aa3a1cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0413100407.1649865359.840934/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0413100407.1649865359.840934/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0413100407.1649865359.840934/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0413100407.1649865359.840934/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220413155559841869-6999'
createTime: '2022-04-13T15:56:01.315332Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-13_08_56_00-18197072409769740729'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0413100407'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-13T15:56:01.315332Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-13_08_56_00-18197072409769740729]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-13_08_56_00-18197072409769740729
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-13_08_56_00-18197072409769740729?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-13_08_56_00-18197072409769740729 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:07.511Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.742Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.827Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.875Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.912Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.955Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:08.993Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.064Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.179Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.219Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.259Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.289Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.328Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.365Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.403Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.439Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.627Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.716Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.779Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:09.813Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-13_08_56_00-18197072409769740729 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:34.805Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:56:54.587Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T15:57:15.155Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.261Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.375Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.407Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.484Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.551Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.628Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T16:02:32.704Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T20:00:29.683Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-13_08_56_00-18197072409769740729.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T20:00:29.717Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T20:00:29.887Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T20:00:30.070Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-13T20:00:30.098Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-13_08_56_00-18197072409769740729 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 22s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/vy3mevs2utzkm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/551/display/redirect?page=changes>
Changes:
[kamil.bregula] [BEAM-14215] Improve argument validation in SnowflakeIO
[benjamin.gonzalez] [BEAM-14013] Add PreCommit Kotlin examples Jenkins Job
[Andrew Pilloud] [BEAM-13151] Support multiple layers of AutoValue nesting
[Heejong Lee] [BEAM-14233] Merge requirements from expanded response for Java External
[benjamin.gonzalez] [BEAM-14013] Add spark, direct, flink runners as triggers for Kotlin
[noreply] [BEAM-13898] Add tests to the pubsubx package. (#17324)
[noreply] [BEAM-14285] Clean up Staticcheck Warnings in io packages (#17336)
[noreply] [BEAM-14187] Fix concurrency issue in IsmReaderImpl (#17201)
[noreply] [BEAM-14288] Skip flaking test
[noreply] Simplify specifying additional dependencies in Go SDK in XLang IOs
[noreply] [BEAM-14240] Clean staticcheck warnings in runner packages (#17340)
[Daniel Oliveira] [BEAM-13538] Workaround to fix go-licenses crash.
------------------------------------------
[...truncated 325.26 KB...]
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.38
Using cached botocore-1.24.38-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2726953 sha256=06aea6276247dea41debc9232f4d5574da2e7a0802ae9a0a515ef52d81af6951
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.38 botocore-1.24.38 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0bc85274d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0bc8527cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0412100426.1649778979.518454/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0412100426.1649778979.518454/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0412100426.1649778979.518454/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0412100426.1649778979.518454/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220412155619519357-6230'
createTime: '2022-04-12T15:56:21.220056Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-12_08_56_20-16021287713786818839'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0412100426'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-12T15:56:21.220056Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-12_08_56_20-16021287713786818839]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-12_08_56_20-16021287713786818839
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-12_08_56_20-16021287713786818839?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-12_08_56_20-16021287713786818839 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:24.670Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.035Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.076Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.116Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.150Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.192Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.229Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.255Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.290Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.314Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.364Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.436Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.543Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.595Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.681Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.738Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.903Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:30.973Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:31.019Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:31.050Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-12_08_56_20-16021287713786818839 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:56:59.697Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:57:00.759Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 1 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:57:00.808Z: JOB_MESSAGE_DETAILED: Resized **** pool to 1, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:57:11.070Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T15:57:34.586Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:43.320Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:43.513Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:43.591Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:43.775Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:43.927Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:44.092Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T16:02:44.247Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:00:32.759Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-12_08_56_20-16021287713786818839.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:00:32.896Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:00:33.150Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:00:33.233Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:00:33.273Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-12_08_56_20-16021287713786818839 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:01:15.037Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:01:15.479Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-12T20:01:15.579Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-12_08_56_20-16021287713786818839 is in state JOB_STATE_CANCELLED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1658, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 5m 58s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/xrgkhiyq6ozpo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/550/display/redirect>
Changes:
------------------------------------------
[...truncated 324.59 KB...]
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.37
Using cached botocore-1.24.37-py3-none-any.whl (8.7 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2726931 sha256=cfdde5f94a2b573de25010729f516be4bf3c1b9cce88a0525abb549f4e053753
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.37 botocore-1.24.37 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1c867f64d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1c867f6cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0411100425.1649691860.032107/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0411100425.1649691860.032107/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0411100425.1649691860.032107/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0411100425.1649691860.032107/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220411154420033020-3710'
createTime: '2022-04-11T15:44:21.276311Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-11_08_44_20-18149929412057679571'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0411100425'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-11T15:44:21.276311Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-11_08_44_20-18149929412057679571]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-11_08_44_20-18149929412057679571
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-11_08_44_20-18149929412057679571?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-11_08_44_20-18149929412057679571 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:23.267Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.209Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.242Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.268Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.301Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.347Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.380Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.413Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.449Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.484Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.516Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.551Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.620Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.682Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.716Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.750Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:24.929Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:25.007Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:25.056Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:25.087Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-11_08_44_20-18149929412057679571 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:44:43.368Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:45:01.062Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:45:01.092Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:45:11.321Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:45:34.760Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:47.880Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:47.955Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:47.991Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:48.050Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:48.098Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:48.141Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T15:50:48.202Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:00:36.911Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-11_08_44_20-18149929412057679571.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:00:36.938Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:00:37.340Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:00:37.420Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:00:37.440Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-11_08_44_20-18149929412057679571 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:01:31.701Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:01:31.859Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-11T20:01:31.916Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 18m 17s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/uat74zxwhf722
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/549/display/redirect?page=changes>
Changes:
[chamikaramj] Re-raise exceptions swallowed in several Python I/O connectors
[noreply] Merge pull request #16928: [BEAM-11971] Re add reverted timer
------------------------------------------
[...truncated 324.15 KB...]
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.37-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.37
Using cached botocore-1.24.37-py3-none-any.whl (8.7 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2726931 sha256=9d7f66b5ca5154177882ce5c78f0d33921bd4b41054b1e8038611ed3e4de1967
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.8
Uninstalling pyparsing-3.0.8:
Successfully uninstalled pyparsing-3.0.8
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.37 botocore-1.24.37 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff15b9944d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff15b994cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0410100427.1649605231.940178/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0410100427.1649605231.940178/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0410100427.1649605231.940178/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0410100427.1649605231.940178/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220410154031941126-9534'
createTime: '2022-04-10T15:40:33.263489Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-10_08_40_32-198240457815369459'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0410100427'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-10T15:40:33.263489Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-10_08_40_32-198240457815369459]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-10_08_40_32-198240457815369459
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-10_08_40_32-198240457815369459?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-10_08_40_32-198240457815369459 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:35.495Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.674Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.702Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.749Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.781Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.838Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.876Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.920Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.959Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:36.991Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.024Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.059Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.092Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.125Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.160Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.194Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.315Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.375Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.466Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:37.536Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-10_08_40_32-198240457815369459 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:40:58.490Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:41:10.751Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 7 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:41:10.777Z: JOB_MESSAGE_DETAILED: Resized **** pool to 7, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:41:21.055Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:41:45.219Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.330Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.408Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.437Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.506Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.581Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.631Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T15:48:25.686Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T20:00:26.904Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-10_08_40_32-198240457815369459.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T20:00:26.933Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T20:00:27.108Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T20:00:27.194Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-10T20:00:27.219Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-10_08_40_32-198240457815369459 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 21m 49s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/gyu3ycb3sdqow
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/548/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-11714] Change spotBugs jenkins config
[Robert Bradshaw] Cleanup docs on Shared.
[Kyle Weaver] Nit: correct description for precommit cron jobs.
[benjamin.gonzalez] [BEAM-11714] Add dummy class for testing
[benjamin.gonzalez] [BEAM-11714] Remove dummy class used for testing
[benjamin.gonzalez] [BEAM-11714] Spotbugs print toJenkins UI precommit_Java17
[noreply] [BEAM-13767] Remove eclipse plugin as it generates a lot of unused tasks
[noreply] [BEAM-10708] Updated beam_sql error message (#17314)
[noreply] [BEAM-14281] add as_deterministic_coder to nullable coder (#17322)
[noreply] Improvements to Beam/Spark quickstart. (#17129)
[chamikaramj] Disable BigQueryIOStorageWriteIT for Runner v2 test suite
------------------------------------------
[...truncated 323.74 KB...]
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.1-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.37
Using cached botocore-1.24.37-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2726702 sha256=e8fb07c55ea5107ce03eac4232db6d292a0474adb10b486c07aa15b6946378c7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.37 botocore-1.24.37 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0a03d8c4d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0a03d8ccb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0409100416.1649519303.236175/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0409100416.1649519303.236175/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0409100416.1649519303.236175/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0409100416.1649519303.236175/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220409154823237102-4647'
createTime: '2022-04-09T15:48:27.049780Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-09_08_48_23-9039427419160848621'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0409100416'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-09T15:48:27.049780Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-09_08_48_23-9039427419160848621]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-09_08_48_23-9039427419160848621
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-09_08_48_23-9039427419160848621?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-09_08_48_23-9039427419160848621 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:29.387Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.332Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.363Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.386Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.412Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.461Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.477Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.510Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.545Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.571Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.595Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.617Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.643Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.672Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.706Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.739Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.869Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.921Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:30.973Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:31.020Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-09_08_48_23-9039427419160848621 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:48:51.817Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:49:05.739Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:49:05.760Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:49:16.015Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:49:37.450Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:44.787Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:44.896Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:44.932Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:45.007Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:45.084Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:45.131Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T15:54:45.193Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T20:00:49.678Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-09_08_48_23-9039427419160848621.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T20:00:49.700Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T20:00:49.805Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T20:00:49.929Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-09T20:00:49.949Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-09_08_48_23-9039427419160848621 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 15s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/qekzxs4bl5aw2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/547/display/redirect?page=changes>
Changes:
[johnjcasey] [BEAM-10529] add java and generic components of nullable xlang tests
[johnjcasey] [BEAM-10529] fix test case
[johnjcasey] [BEAM-10529] add coders and typehints to support nullable xlang coders
[johnjcasey] [BEAM-10529] update external builder to support nullable coder
[johnjcasey] [BEAM-10529] clean up coders.py
[johnjcasey] [BEAM-10529] add coder translation test
[johnjcasey] [BEAM-10529] add additional check to typecoder to not accidentally
[johnjcasey] [BEAM-10529] add test to retrieve nullable coder from typehint
[johnjcasey] [BEAM-10529] run spotless
[johnjcasey] [BEAM-10529] add go nullable coder
[johnjcasey] [BEAM-10529] cleanup extra println
[johnjcasey] [BEAM-10529] improve comments, clean up python
[bulat.safiullin] [BEAM-13992] [Website] update Contribute/Code Contribution Guide page
[bulat.safiullin] [BEAM-13992] [Website] change text, transfer tag a
[bulat.safiullin] [BEAM-13992] [Website] change code tags
[bulat.safiullin] [BEAM-13992] [Website] change text
[bulat.safiullin] [BEAM-13992] [Website] change text and links, add empty lines
[bulat.safiullin] [BEAM-13991] [Website] change links, add contribute file
[bulat.safiullin] [BEAM-13991] [Website] add content, add styles
[bulat.safiullin] [BEAM-13991] [Website] add images, add styles, delete spaces
[bulat.safiullin] [BEAM-13991] [Website] change url and aliases, delete bullet points
[bulat.safiullin] [BEAM-13991] [Website] add empty line
[bulat.safiullin] [BEAM-13992] [Website] change links, change text
[bulat.safiullin] [BEAM-13992] [Website] change links, add text, add dots
[bulat.safiullin] [BEAM-13992] [Website] change links, change text
[bulat.safiullin] [BEAM-13991] [Website] change styles, change quotes
[bulat.safiullin] [BEAM-13991] [Website] change link color
[bulat.safiullin] [BEAM-13992] [Website] change text, delete whitespace
[bulat.safiullin] [BEAM-13991] [Website] change text
[bulat.safiullin] [BEAM-13992] [Website] update text
[bulat.safiullin] [BEAM-13991] [Website] added changes from PR 13992, changed get-starting
[shivrajw] [BEAM-14236] Parquet IO support for list to conform with Apache Parquet
[chamikaramj] Sets 'sdk_harness_container_images' property for all Dataflow jobs -
[chamikaramj] Sets 'sdk_harness_container_images' property for all Dataflow jobs -
[mmack] [BEAM-14104] Support shard aware aggregation in Kinesis writer.
[noreply] [BEAM-13015] Lookup the container for the step once when registering
[noreply] [BEAM-14175] Log read loop abort at debug rather than error (#17183)
[noreply] [BEAM-11745] Fix author list rendering (#17308)
[noreply] [BEAM-14144] Record JFR profiles when GC thrashing is detected (#17151)
[noreply] Factors enable_prime flag in when checking use_unified_worker conditions
[noreply] [BEAM-11104] Add ProcessContinuation type to Go SDK (#17265)
[noreply] BEAM-13939: Restructure Protos to fix namespace conflicts (#16961)
[noreply] [BEAM-14270] Mark {Snowflake/BigQuery}Services as @Internal (#17309)
[noreply] [BEAM-13901] Add unit tests for graphx/cogbk.go
[noreply] [BEAM-14259, BEAM-14266] Remove unused function, replace use of ptypes
[noreply] [BEAM-14274] Fix staticcheck warnings in pipelinex (#17311)
[noreply] [BEAM-13857] Switched Go IT script to using Go flags for expansion
[noreply] Update python beam-master container image. (#17313)
------------------------------------------
[...truncated 324.42 KB...]
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.35-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.36
Using cached botocore-1.24.36-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2726519 sha256=633a700a0f38f27ee3e82fe3867ea8e90a7c8c3acf13af486d756bf0bc793ed3
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.36 botocore-1.24.36 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.2 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220407" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff4ff4b93b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff4ff4b9b90> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0408100419.1649431875.814860/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0408100419.1649431875.814860/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0408100419.1649431875.814860/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0408100419.1649431875.814860/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220408153115815821-5935'
createTime: '2022-04-08T15:31:19.606045Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-08_08_31_16-14813728799684243475'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0408100419'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-08T15:31:19.606045Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-08_08_31_16-14813728799684243475]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-08_08_31_16-14813728799684243475
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-08_08_31_16-14813728799684243475?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-08_08_31_16-14813728799684243475 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:22.346Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.358Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.432Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.487Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.546Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.617Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.645Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.683Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.713Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.750Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.819Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.876Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.937Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:23.974Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.022Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.057Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.191Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.251Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.296Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:24.328Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-08_08_31_16-14813728799684243475 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:50.440Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:55.606Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:31:55.687Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:32:05.939Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:32:28.690Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.253Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.349Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.381Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.454Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.511Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.578Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T15:38:55.651Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-08_08_31_16-14813728799684243475 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T20:00:39.244Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-08_08_31_16-14813728799684243475.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T20:00:39.294Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T20:00:39.495Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T20:00:39.626Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-08T20:00:39.651Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 31m 22s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/kv637ol63pvxi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/546/display/redirect?page=changes>
Changes:
[noreply] Avoid pr-bot state desync (#17299)
[noreply] [BEAM-14259] Clean up staticcheck warnings in the exec package (#17285)
[noreply] Minor: Prefer registered schema in SQL docs (#17298)
[Kyle Weaver] [BEAM-14262] Update plugins for Dockerized Jenkins.
[Kyle Weaver] Add ansicolor and ws-cleanup plugins.
[noreply] [Playground] add meta tags (#17207)
[noreply] fixes golint and deprecated issues in recent Go SDK import (#17304)
[noreply] [BEAM-14266] Replace deprecated ptypes package uses (#17302)
[noreply] [BEAM-11936] Fix rawtypes warnings in SnowflakeIO (#17257)
[noreply] Merge pull request #17262: [BEAM-14244] Use the supplied output
------------------------------------------
[...truncated 321.76 KB...]
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.35-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.35-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.35
Using cached botocore-1.24.35-py3-none-any.whl (8.7 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2712736 sha256=af3541ca5d1d6fa3af140581356c090f1f1fbd56287f114d8438d64c98a0a111
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.35 botocore-1.24.35 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.12.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.2 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.35 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f004f5e5950> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f004f5e6170> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0407100428.1649346632.640347/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0407100428.1649346632.640347/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0407100428.1649346632.640347/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0407100428.1649346632.640347/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220407155032641420-2633'
createTime: '2022-04-07T15:50:33.756581Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-07_08_50_33-9888555807101221406'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0407100428'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-07T15:50:33.756581Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-07_08_50_33-9888555807101221406]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-07_08_50_33-9888555807101221406
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-07_08_50_33-9888555807101221406?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-07_08_50_33-9888555807101221406 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:47.130Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:50.974Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.008Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.027Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.051Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.097Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.118Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.145Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.188Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.214Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.233Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.258Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.286Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.335Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.357Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.380Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.521Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.595Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.636Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:50:51.659Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-07_08_50_33-9888555807101221406 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:51:09.330Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:51:25.914Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:51:25.950Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:51:36.163Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:51:59.022Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.502Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.544Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.570Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.617Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.682Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.731Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T15:58:32.816Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:00:30.806Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-07_08_50_33-9888555807101221406.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:00:30.840Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:00:31.004Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:00:31.250Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:00:31.295Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-07_08_50_33-9888555807101221406 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:01:26.335Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:01:26.373Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-07T20:01:26.420Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 12m 10s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/mlvtu3b2ufi4g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #545
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/545/display/redirect?page=changes>
Changes:
[bingyeli] update query
[Robert Bradshaw] [BEAM-14250] Fix coder registration for types defined in __main__.
[johnjcasey] [BEAM-14256] update SpEL dependency to 5.3.18.RELEASE
[johnjcasey] [BEAM-14256] remove .RELEASE
[dannymccormick] Fix dependency issue causing failures
[Kyle Weaver] [BEAM-9649] Add region option to Mongo Dataflow test.
[noreply] Allow get_coder(None).
[noreply] [BEAM-13015] Disable retries for fnapi grpc channels which otherwise
[noreply] [BEAM-13952] Sickbay
[noreply] BEAM-14235 parquetio module does not parse PEP-440 compliant Pyarrow
[noreply] [Website] Contribution guide page indent bug fix (#17287)
[noreply] [BEAM-10976] Document go sdk bundle finalization (#17048)
[noreply] [BEAM-13829] Expose status API from Go SDK Harness (#16957)
------------------------------------------
[...truncated 320.53 KB...]
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.34-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.3-py2.py3-none-any.whl (35 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.34
Using cached botocore-1.24.34-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2712736 sha256=fc168e7af2c47b631415e9ec68cdc5e9b02391397702574d49e88ddd67d71a3f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.34 botocore-1.24.34 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.1 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.1 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1cb50909e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1cb5091200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0406100428.1649258533.350636/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0406100428.1649258533.350636/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0406100428.1649258533.350636/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0406100428.1649258533.350636/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220406152213351543-4111'
createTime: '2022-04-06T15:22:14.262615Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-06_08_22_13-15385375392523016528'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0406100428'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-06T15:22:14.262615Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-06_08_22_13-15385375392523016528]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-06_08_22_13-15385375392523016528
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-06_08_22_13-15385375392523016528?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-06_08_22_13-15385375392523016528 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:16.486Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.347Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.386Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.421Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.456Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.494Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.529Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.561Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.591Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.620Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.652Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.674Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.706Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.733Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.759Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.784Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:18.972Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:19.040Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:19.081Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:19.102Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-06_08_22_13-15385375392523016528 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:22:27.692Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:23:03.103Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:23:27.045Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:40.785Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:40.874Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:40.908Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:40.961Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:41.042Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:41.102Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T15:28:41.181Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:00:35.486Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-06_08_22_13-15385375392523016528.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:00:35.510Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:00:35.676Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:00:35.863Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:00:35.915Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-06_08_22_13-15385375392523016528 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:01:30.345Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 10 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:01:30.383Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-06T20:01:30.412Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 40m 13s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/exlxghx2rdpbg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/544/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-8970] Add docs to run wordcount example on portable Spark Runner
[Kiley Sok] Update python container version
[benjamin.gonzalez] [BEAM-8970] Add period to end of sentence
[Kyle Weaver] Add self-descriptive message for expected errors.
[noreply] Add --dataflowServiceOptions=enable_prime to useUnifiedWorker conditions
[noreply] [BEAM-10529] nullable xlang coder (#16923)
[noreply] Fix go fmt break in core/typex/special.go (#17266)
[noreply] [BEAM-5436] Add doc page on Go cross compilation. (#17256)
[noreply] Pr-bot Don't count all reviews as approvals (#17269)
[noreply] Fix postcommits (#17263)
[noreply] [BEAM-14241] Address staticcheck warnings in boot.go (#17264)
[noreply] [BEAM-14157] GrpcWindmillServer: Use stream specific boolean to do
[noreply] [BEAM-10582] Allow (and test) pyarrow 7 (#17229)
[noreply] [BEAM-13519] Solve race issues when the server responds with an error
------------------------------------------
[...truncated 321.84 KB...]
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.33
Using cached botocore-1.24.33-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2712594 sha256=10ba8919ae4d322e5c633ab469c75ace2ae26c33b0dc1ff531b6b2636b74ff04
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.33 botocore-1.24.33 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.1 google-cloud-core-1.7.2 google-cloud-datastore-1.15.4 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.1 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.3 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220331" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5a417db9e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5a417dc200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0405100429.1649174070.847688/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0405100429.1649174070.847688/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0405100429.1649174070.847688/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0405100429.1649174070.847688/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220405155430848640-5601'
createTime: '2022-04-05T15:54:33.644996Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-05_08_54_31-9452284332100592192'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0405100429'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-05T15:54:33.644996Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-05_08_54_31-9452284332100592192]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-05_08_54_31-9452284332100592192
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-05_08_54_31-9452284332100592192?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-05_08_54_31-9452284332100592192 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:36.875Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.183Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.224Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.255Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.291Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.332Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.373Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.415Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.454Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.486Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.555Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.604Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.676Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.721Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.753Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:38.790Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-05_08_54_31-9452284332100592192 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:39.162Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:39.230Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:39.274Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:39.301Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:54:56.231Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:55:08.455Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 4 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:55:08.491Z: JOB_MESSAGE_DETAILED: Resized **** pool to 4, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:55:18.699Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T15:55:41.526Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.408Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.535Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.569Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.636Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.685Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.750Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T16:02:21.816Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T20:00:41.026Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-05_08_54_31-9452284332100592192.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T20:00:41.098Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T20:00:41.247Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T20:00:41.455Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-05T20:00:41.480Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-05_08_54_31-9452284332100592192 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 8m 7s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/7bcfiiwlm546w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/543/display/redirect>
Changes:
------------------------------------------
[...truncated 322.00 KB...]
Using cached google_cloud_bigquery_storage-2.13.0-py2.py3-none-any.whl (179 kB)
Collecting google-cloud-core<2,>=0.28.1
Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.32
Using cached botocore-1.24.32-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (62.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2711991 sha256=1c7b11e0d7ff0d0aeac625398c0dd8e4e6c73e337a527392823e49c74267d7f5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.32 botocore-1.24.32 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f55e12bb170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f55e12bb950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0404100413.1649088062.292434/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0404100413.1649088062.292434/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0404100413.1649088062.292434/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0404100413.1649088062.292434/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220404160102293340-6890'
createTime: '2022-04-04T16:01:03.763415Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-04_09_01_02-1660966995478529485'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0404100413'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-04T16:01:03.763415Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-04_09_01_02-1660966995478529485]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-04_09_01_02-1660966995478529485
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-04_09_01_02-1660966995478529485?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-04_09_01_02-1660966995478529485 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:25.899Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.713Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.765Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.797Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.830Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.869Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.907Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:26.948Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.001Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.034Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.066Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.131Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.166Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.217Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.247Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.281Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.411Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.478Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.521Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:27.554Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-04_09_01_02-1660966995478529485 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:01:42.240Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:02:10.553Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:02:34.516Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.223Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.346Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.366Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.411Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.465Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.522Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T16:09:10.598Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T20:00:26.626Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-04_09_01_02-1660966995478529485.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T20:00:26.654Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T20:00:26.853Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T20:00:26.916Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-04T20:00:26.941Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-04_09_01_02-1660966995478529485 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 1m 16s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/i3sxqspfriuq6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/542/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14143] Simplifies the ExternalPythonTransform API (#17101)
------------------------------------------
[...truncated 320.70 KB...]
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.34-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.0-py2.py3-none-any.whl (33 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.32
Using cached botocore-1.24.32-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.3.1)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.8.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2711991 sha256=0cd8ff78dbff253ca280dc0c8e98423bb7b2103fe4479d5f6ede9f46ef6eab71
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.32 botocore-1.24.32 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.0 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc5ba2c9170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc5ba2c9950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0403100437.1649001383.008532/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0403100437.1649001383.008532/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0403100437.1649001383.008532/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0403100437.1649001383.008532/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220403155623009451-1170'
createTime: '2022-04-03T15:56:23.953917Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-03_08_56_23-6350157951682546938'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0403100437'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-03T15:56:23.953917Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-03_08_56_23-6350157951682546938]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-03_08_56_23-6350157951682546938
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-03_08_56_23-6350157951682546938?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-03_08_56_23-6350157951682546938 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:29.759Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.609Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.637Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.667Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.699Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.732Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.760Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.783Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.817Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.853Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.876Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.911Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:35.978Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.022Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.048Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.069Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.200Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.278Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.316Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:36.347Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-03_08_56_23-6350157951682546938 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:56:47.494Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:57:11.535Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 6 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:57:11.570Z: JOB_MESSAGE_DETAILED: Resized **** pool to 6, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:57:21.773Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T15:57:42.857Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.596Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.696Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.729Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.847Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.896Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:11.961Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T16:04:12.025Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T20:00:36.622Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-03_08_56_23-6350157951682546938.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T20:00:36.648Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T20:00:36.803Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T20:00:36.890Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-03T20:00:36.935Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-03_08_56_23-6350157951682546938 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 9s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/6x4piw2wpha5k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #541
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/541/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14133] Fix potential NPE in BigQueryServicesImpl.getErrorInfo
[Robert Bradshaw] Revert "Revert "[BEAM-14038] Auto-startup for Python expansion service.
[Robert Bradshaw] Skip failing test for now.
[Kyle Weaver] [BEAM-14225] load balance jenkins jobs
[noreply] [BEAM-14153] Reshuffled Row Coder PCollection used as Side Input cause
[noreply] delint go sdk (#17247)
[Heejong Lee] add test
[noreply] Merge pull request #16841 from [BEAM-8823] Make FnApiRunner work by
[noreply] [BEAM-14192] Update legacy container version (#17210)
[noreply] Fix mishandling of API with BQIO (#17211)
[noreply] [BEAM-14221] Update documentation with Flink on Dataproc features
[Kiley Sok] Revert "[BEAM-14190] Python sends dataflow schema field"
------------------------------------------
[...truncated 321.46 KB...]
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.34-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.0-py2.py3-none-any.whl (33 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.32
Using cached botocore-1.24.32-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.3.1)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2711991 sha256=3075a34e865f022f3515da4987c8b6961b0cfbee2fcf732fc3397919ada31ccf
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.32 botocore-1.24.32 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.0 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fdbd5b26170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fdbd5b26950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0402100419.1648915238.939593/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0402100419.1648915238.939593/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0402100419.1648915238.939593/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0402100419.1648915238.939593/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220402160038940524-2095'
createTime: '2022-04-02T16:00:39.960949Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-02_09_00_39-12055322680998393951'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0402100419'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-02T16:00:39.960949Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-02_09_00_39-12055322680998393951]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-02_09_00_39-12055322680998393951
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-02_09_00_39-12055322680998393951?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-02_09_00_39-12055322680998393951 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:43.333Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.484Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.566Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.600Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.633Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.693Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.723Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.760Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.795Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.823Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.859Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.881Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.921Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.953Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:46.989Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:47.024Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:47.203Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:47.279Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:47.319Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:00:47.356Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-02_09_00_39-12055322680998393951 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:01:07.722Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:01:16.512Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:01:16.554Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:01:26.816Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:01:47.645Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.274Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.479Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.512Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.568Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.632Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.692Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T16:08:05.747Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T20:00:36.803Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-02_09_00_39-12055322680998393951.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T20:00:36.830Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T20:00:36.902Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T20:00:37.267Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-02T20:00:37.289Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-02_09_00_39-12055322680998393951 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 1m 56s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/sx2gnobok6ly6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/540/display/redirect?page=changes>
Changes:
[bulat.safiullin] [BEAM-14164] [Website] change styles
[Andrew Pilloud] [BEAM-14190] Python sends dataflow schema field
[noreply] [BEAM-14179] Fix possibly null value
[noreply] [BEAM-12815] Try to fix flaky Flink Post Commit (#17227)
[noreply] Add a portable job server that proxies the Dataflow backend. (#17189)
[noreply] [BEAM-14130] Implement JupyterLab extension for managing Dataproc
[Andrew Pilloud] [BEAM-13741] Remove forced calcite dependency from BaseBeamTable
[noreply] [BEAM-13951] Update release guide with pointers on updating
------------------------------------------
[...truncated 322.01 KB...]
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.34-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.5.0-py2.py3-none-any.whl (33 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.31
Using cached botocore-1.24.31-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.3.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2703619 sha256=7b33540618fb2ea5859ae4609b0da7c2d7d714edfcbf266989912cc3adee472e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.1 azure-storage-blob-12.11.0 boto3-1.21.31 botocore-1.24.31 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.34 tenacity-5.1.5 testcontainers-3.5.0 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6f8dbc60e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6f8dbc68c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0401100411.1648829162.439698/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0401100411.1648829162.439698/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0401100411.1648829162.439698/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0401100411.1648829162.439698/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220401160602441701-5578'
createTime: '2022-04-01T16:06:03.698937Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-04-01_09_06_03-12319873641892618664'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0401100411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-04-01T16:06:03.698937Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-04-01_09_06_03-12319873641892618664]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-04-01_09_06_03-12319873641892618664
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-01_09_06_03-12319873641892618664?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-01_09_06_03-12319873641892618664 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:06.350Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:08.788Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:08.851Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:08.908Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:08.934Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:08.975Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.010Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.054Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.085Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.198Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.241Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.294Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.408Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.525Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.583Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.635Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.843Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.944Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:09.978Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:10.011Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-01_09_06_03-12319873641892618664 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:39.115Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:06:57.328Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:07:18.754Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.598Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.687Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.711Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.790Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.847Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.910Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T16:13:48.977Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T20:00:28.008Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-04-01_09_06_03-12319873641892618664.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T20:00:28.044Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T20:00:28.207Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T20:00:28.269Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-04-01T20:00:28.299Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-04-01_09_06_03-12319873641892618664 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 56m 24s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/wkb2he7td7gzi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #539
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/539/display/redirect?page=changes>
Changes:
[brachipa] [BEAM-14094]Fix null pointer exception in HllCountInitFn
[brachipa] [BEAM-14094]Fix null pointer exception in HllCountInitFn
[egalpin] [BEAM-14003] Adds compat for Elasticsearch 8.0.0
[egalpin] [BEAM-13136] Removes support for Elasticsearch 2.x
[noreply] Merge pull request #17149 from [BEAM-13883] [Playground] Increase test
[Kiley Sok] ignore test
[noreply] [BEAM-13948] Add unstack(), a non-deferred column operation on
[noreply] [BEAM-10976] Bundle finalization: E2E support (#17045)
------------------------------------------
[...truncated 322.25 KB...]
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.30-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.30
Using cached botocore-1.24.30-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2698067 sha256=0082939bc32245bba49ce7c3fc31f8cf21173808ec570ab82825d30a0710ae17
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.11.0 boto3-1.21.30 botocore-1.24.30 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.33 tenacity-5.1.5 testcontainers-3.5.0 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4586e37e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4586e38680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0331100409.1648743028.497580/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0331100409.1648743028.497580/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0331100409.1648743028.497580/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0331100409.1648743028.497580/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220331161028498661-7591'
createTime: '2022-03-31T16:10:29.507197Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-31_09_10_29-13852572926766744200'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0331100409'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-31T16:10:29.507197Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-31_09_10_29-13852572926766744200]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-31_09_10_29-13852572926766744200
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-31_09_10_29-13852572926766744200?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-31_09_10_29-13852572926766744200 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:32.475Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.561Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.622Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.647Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.678Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.737Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.798Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.821Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.842Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.902Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.938Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:33.994Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.022Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.053Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.086Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.120Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.277Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.344Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.390Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:34.421Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-31_09_10_29-13852572926766744200 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:10:49.587Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:11:09.733Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:11:09.768Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:11:20.049Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:11:41.007Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.517Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.579Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.612Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.732Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.793Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.866Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T16:18:18.932Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T20:00:50.978Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-31_09_10_29-13852572926766744200.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T20:00:51.003Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T20:00:51.741Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T20:00:51.781Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-31T20:00:51.808Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-31_09_10_29-13852572926766744200 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 52m 18s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/7w3uqj6s3t5x4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #538
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/538/display/redirect?page=changes>
Changes:
[vachan] Update display data to include BQ information.
[noreply] Revert "[BEAM-14084] iterable_input_value_types changed from list to
[chamikaramj] Convert URLs to local jars when constructing filesToStage
[Valentyn Tymofieiev] Ensure the removed option prebuild_sdk_container_base_image not used on
[noreply] [BEAM-13314]Revise recommendations to manage Python pipeline
[noreply] Merge pull request #17202 from [BEAM-14194]: Disallow autoscaling for
[noreply] Merge pull request #17080 from [BEAM-13880] [Playground] Increase test
[noreply] Merge pull request #17050 from [BEAM-13877] [Playground] Increase test
[noreply] [BEAM-14200] Improve SamzaJobInvoker extensibility (#17212)
[noreply] Merge pull request #17148 from [BEAM-14042] [playground] Scroll imports
[noreply] [BEAM-13918] Increase datastoreio go sdk unit test coverage (#17173)
[noreply] Merge pull request #16819: [BEAM-13806] Adding test suite for Go x-lang
------------------------------------------
[...truncated 324.39 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2696577 sha256=47df6784f2bd0d8f747628ee3dc9756a49cfe982f7f42badc4b1194be432457e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.11.0 boto3-1.21.29 botocore-1.24.29 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.3 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fce0fab4e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fce0fab5680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0330100359.1648656208.522552/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0330100359.1648656208.522552/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0330100359.1648656208.522552/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0330100359.1648656208.522552/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220330160328523620-3282'
createTime: '2022-03-30T16:03:29.476251Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-30_09_03_29-14039983512698260999'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0330100359'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-30T16:03:29.476251Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-30_09_03_29-14039983512698260999]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-30_09_03_29-14039983512698260999
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-30_09_03_29-14039983512698260999?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-30_09_03_29-14039983512698260999 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:32.180Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.124Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.380Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.439Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.502Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.588Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.628Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.670Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.712Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.745Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.808Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.851Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:34.918Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:35.042Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:35.212Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:35.501Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:36.432Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:36.955Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:37.193Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:03:37.304Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-30_09_03_29-14039983512698260999 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:04:04.084Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:04:06.434Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:04:06.465Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:04:16.768Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:04:37.207Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.065Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.151Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.173Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.239Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.301Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.391Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T16:11:13.467Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T20:09:27.740Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-30_09_03_29-14039983512698260999 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T20:09:28.736Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T20:09:28.785Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-30T20:09:28.832Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 7m 51s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/ewq6thq6bbx6c
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@68511f6b:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:994)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy132.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException
at hudson.remoting.Channel.close(Channel.java:1477)
at hudson.remoting.Channel.close(Channel.java:1454)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:894)
at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:108)
at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:774)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
FATAL: Channel "hudson.remoting.Channel@68511f6b:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
java.io.IOException
at hudson.remoting.Channel.close(Channel.java:1477)
at hudson.remoting.Channel.close(Channel.java:1454)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:894)
at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:108)
at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:774)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@68511f6b:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:994)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1148)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #537
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/537/display/redirect?page=changes>
Changes:
[hengfeng] feat: remove the metadata table after the pipeline finishes
[thiagotnunes] test: add test for metadata table dropping
[noreply] Minor: Add warning about pubsub client to Beam 2.36.0 blog (#17188)
[noreply] [BEAM-14177] Fix GBK re-iteration caching for portable runners. (#17184)
[noreply] Merge pull request #17187: [BEAM-14181] Make sure to evict connections
[noreply] Only reset transform.label if it is correctly assigned (#17192)
[noreply] [BEAM-12641] Use google-auth instead of oauth2client for GCP auth
[Robert Bradshaw] [BEAM-14163] Fix typo in single core per container logic.
[thiagotnunes] test: disable SpannerIO.readChangeStream test
[noreply] Merge pull request #17164 from [BEAM-14140][Playground] Fix Deploy
[noreply] Merge pull request #16855 from [BEAM-13938][Playground] Increase test
------------------------------------------
[...truncated 321.69 KB...]
Collecting google-cloud-bigquery-storage>=2.6.3
Using cached google_cloud_bigquery_storage-2.13.0-py2.py3-none-any.whl (179 kB)
Collecting google-cloud-core<2,>=0.28.1
Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.28
Using cached botocore-1.24.28-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.2-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2696201 sha256=d1fd75cde236761c634cb678f4e23a704997445a20804c72f4f20d33017491fa
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.11.0 boto3-1.21.28 botocore-1.24.28 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.2 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa6b8d62950> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa6b8d63170> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0329100415.1648568508.692106/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0329100415.1648568508.692106/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0329100415.1648568508.692106/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0329100415.1648568508.692106/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220329154148693061-8043'
createTime: '2022-03-29T15:41:49.625715Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-29_08_41_49-14377698893574678546'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0329100415'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-29T15:41:49.625715Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-29_08_41_49-14377698893574678546]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-29_08_41_49-14377698893574678546
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-29_08_41_49-14377698893574678546?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-29_08_41_49-14377698893574678546 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:52.331Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.278Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.361Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.407Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.440Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.503Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.572Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.608Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.639Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.663Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.697Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.731Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.801Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.835Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.904Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:53.925Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:54.133Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:54.213Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:54.247Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:41:54.278Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-29_08_41_49-14377698893574678546 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:42:24.247Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:42:39.769Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:43:04.150Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:21.739Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:21.912Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:21.999Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:22.143Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:22.212Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:22.312Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T15:49:22.388Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T20:02:01.121Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-29_08_41_49-14377698893574678546.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T20:02:01.182Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T20:02:02.083Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T20:02:02.171Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-29T20:02:02.195Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-29_08_41_49-14377698893574678546 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 22m 11s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/4gustpmulecz4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #536
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/536/display/redirect>
Changes:
------------------------------------------
[...truncated 322.66 KB...]
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.27
Using cached botocore-1.24.27-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.2.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695994 sha256=aa7d3d4714978fa6f424b09597e8995c0e4e52b6003bcb3495d84a6b0bb07f31
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.27 botocore-1.24.27 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.7.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f89e4a06ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f89e4a07710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0328100408.1648483930.660542/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0328100408.1648483930.660542/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0328100408.1648483930.660542/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0328100408.1648483930.660542/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220328161210661486-9374'
createTime: '2022-03-28T16:12:12.435732Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-28_09_12_11-17288425670859789148'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0328100408'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-28T16:12:12.435732Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-28_09_12_11-17288425670859789148]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-28_09_12_11-17288425670859789148
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-28_09_12_11-17288425670859789148?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-28_09_12_11-17288425670859789148 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.001Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.735Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.766Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.793Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.828Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.869Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.909Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:17.974Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.002Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.025Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.065Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.097Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.135Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.170Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.195Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.338Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.427Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.501Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:18.533Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-28_09_12_11-17288425670859789148 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:12:38.640Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:13:05.199Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:13:36.156Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.371Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.439Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.467Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.535Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.626Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.704Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T16:20:03.772Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T20:00:36.706Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-28_09_12_11-17288425670859789148.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T20:00:36.736Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T20:00:36.876Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T20:00:36.941Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-28T20:00:36.967Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-28_09_12_11-17288425670859789148 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 50m 22s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/v6r6kyt2tqhyc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #535
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/535/display/redirect>
Changes:
------------------------------------------
[...truncated 322.80 KB...]
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.27
Using cached botocore-1.24.27-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.1.1)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695994 sha256=6b5c4677b13ea4d14a00e2f4f5a0d80e3815271d5da8d3357e00e8d0ae7b7cf1
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.27 botocore-1.24.27 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fdaf9595a70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fdaf9596290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0327100411.1648396871.964545/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0327100411.1648396871.964545/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0327100411.1648396871.964545/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0327100411.1648396871.964545/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220327160111965489-5953'
createTime: '2022-03-27T16:01:15.711170Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-27_09_01_12-3504988434895464566'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0327100411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-27T16:01:15.711170Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-27_09_01_12-3504988434895464566]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-27_09_01_12-3504988434895464566
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-27_09_01_12-3504988434895464566?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-27_09_01_12-3504988434895464566 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:19.395Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:20.597Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:20.862Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:20.995Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.056Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.135Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.177Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.212Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.254Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.287Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.332Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.362Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.404Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.439Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.469Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.502Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.755Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.819Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.904Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:21.934Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-27_09_01_12-3504988434895464566 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:50.668Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:57.197Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:01:57.257Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:02:07.521Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:02:07.554Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:02:17.838Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:02:29.280Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:01.627Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:01.748Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:01.804Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:01.881Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:01.937Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:02.014Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T16:09:02.095Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T20:00:40.372Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-27_09_01_12-3504988434895464566.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T20:00:40.399Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T20:00:40.513Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T20:00:40.950Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-27T20:00:40.970Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-27_09_01_12-3504988434895464566 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 1m 25s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/gaidiph4c5qte
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #534
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/534/display/redirect?page=changes>
Changes:
[ryanthompson591] iterable_input_value_types will now be an iterable, I don't anticipate
[marco.robles] [BEAM-8218] PulsarIO Connector
[benjamin.gonzalez] [BEAM-12572] Change examples jobs to run as cron jobs
[benjamin.gonzalez] [BEAM-12572] SpotlessApply
[Robert Bradshaw] [BEAM-14171] More explicit asserts in CoGBKResult.
[Robert Bradshaw] Add some comments.
[noreply] [BEAM-14160] Parse filesToStage in Java expansion service (#17167)
[chamikaramj] Mapped JOB_STATE_RESOURCE_CLEANING_UP to RESOURCE_CLEANING_UP in Python
[noreply] Explicitly import estimator from tensorflow (#17168)
------------------------------------------
[...truncated 321.23 KB...]
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.27
Using cached botocore-1.24.27-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.1.1)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695994 sha256=86b23d9ecf9f50bcf972e361d0346ca64563e3c4ecb2e444167e7ef1f783346d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.27 botocore-1.24.27 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f583109ba70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f583109d290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0326100414.1648310934.635237/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0326100414.1648310934.635237/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0326100414.1648310934.635237/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0326100414.1648310934.635237/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220326160854636185-4840'
createTime: '2022-03-26T16:08:55.919153Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-26_09_08_55-5690370374731752119'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0326100414'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-26T16:08:55.919153Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-26_09_08_55-5690370374731752119]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-26_09_08_55-5690370374731752119
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-26_09_08_55-5690370374731752119?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-26_09_08_55-5690370374731752119 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.026Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.759Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.812Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.833Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.858Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.885Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.912Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.942Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.966Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:58.999Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.033Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.068Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.104Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.135Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.161Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.191Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.371Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.439Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.478Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:08:59.514Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-26_09_08_55-5690370374731752119 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:09:20.664Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:09:24.904Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:09:55.531Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.101Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.158Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.179Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.236Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.288Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.348Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T16:16:26.414Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T20:00:34.353Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-26_09_08_55-5690370374731752119.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T20:00:34.376Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-26_09_08_55-5690370374731752119 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T20:00:34.544Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T20:00:34.651Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-26T20:00:34.675Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1633, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 53m 32s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/rr3miszcjd4le
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #533
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/533/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-14139] Drop support for Flink 1.11.
[Kyle Weaver] [BEAM-14139] Remove obsolete reference to Flink 1.11.
[Kyle Weaver] [BEAM-14139] Update list of supported Flink versions.
[Kyle Weaver] [BEAM-14139] Update CHANGES.md
[noreply] [BEAM-14157] Don't call requestObserver.onNext on a closed windmill
[noreply] Minor: Make IOTypeHints a real NamedTuple (#17174)
[noreply] [BEAM-14172] Update tox.ini for pydocs (#17176)
[noreply] [BEAM-14065] Upgrade vendored bytebuddy to version 1.12.8 (#17028)
------------------------------------------
[...truncated 321.23 KB...]
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.26
Using cached botocore-1.24.26-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (61.0.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695940 sha256=2735cd5b6a8e880e1ff3526a38737086080ee2b9326a135f0bdb6a17d00aa1fe
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.26 botocore-1.24.26 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f974f265a70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f974f26a290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0325100414.1648224860.234326/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0325100414.1648224860.234326/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0325100414.1648224860.234326/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0325100414.1648224860.234326/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220325161420235287-1459'
createTime: '2022-03-25T16:14:21.804872Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-25_09_14_20-12175644957170057418'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0325100414'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-25T16:14:21.804872Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-25_09_14_20-12175644957170057418]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-25_09_14_20-12175644957170057418
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-25_09_14_20-12175644957170057418?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-25_09_14_20-12175644957170057418 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:24.336Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.522Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.566Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.604Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.659Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.704Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.740Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.773Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.801Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.933Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:25.992Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.041Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.092Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.120Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.146Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.175Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.349Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.430Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.480Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:26.510Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-25_09_14_20-12175644957170057418 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:54.939Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:55.901Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:14:55.931Z: JOB_MESSAGE_DETAILED: Resized **** pool to 5, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:15:06.134Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:15:26.671Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.010Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.091Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.118Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.201Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.247Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.302Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T16:21:56.359Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T20:00:35.835Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-25_09_14_20-12175644957170057418.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T20:00:35.861Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T20:00:35.999Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T20:00:36.156Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-25T20:00:36.197Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-25_09_14_20-12175644957170057418 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 48m 13s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/fp73fkp5264py
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #532
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/532/display/redirect?page=changes>
Changes:
[bulat.safiullin] [BEAM-13976] [Website] update homepage
[bulat.safiullin] [BEAM-13976] [Website] update homepage, add logo
[bulat.safiullin] [BEAM-13976] [Website] update text
[bulat.safiullin] [BEAM-13976] [Website] Update Community landing page
[bulat.safiullin] [BEAM-13979] [Website] Update Community/Contact us page
[bulat.safiullin] [BEAM-13979] [Website] update title
[bulat.safiullin] [BEAM-13979] [Website] delete space
[bulat.safiullin] [BEAM-13979] [Website] add Beam Playground
[bulat.safiullin] [BEAM-13976] [Website] delete Beam Playground
[bulat.safiullin] [BEAM-13976] [Website] change navbar css links rules, delete links from
[bulat.safiullin] [BEAM-13977] [Website] delete available-contact-channels on mobile
[bulat.safiullin] [BEAM-13976] [Website] change padding size between the sections
[bulat.safiullin] [BEAM-13976] [Website] change title to capital letters
[bulat.safiullin] [BEAM-13976] [Website] change title
[bulat.safiullin] [BEAM-14040] [Website] create new page, add link
[bulat.safiullin] [BEAM-13977] [Website] change title
[bulat.safiullin] [BEAM-13979] [Website] change text
[bulat.safiullin] [BEAM-13976] [Website] change text
[bulat.safiullin] [BEAM-13977] [Website] change text, add capital letters
[bulat.safiullin] [BEAM-13976] [Website] add playground sass, change text-align
[bulat.safiullin] [BEAM-14040] [Website] add io connectors table
[bulat.safiullin] [BEAM-13976] [Website] add playground section, add empty line
[bulat.safiullin] [BEAM-14040] [Website] add overflow to css, add table content
[bulat.safiullin] [BEAM-14040] [Website] change ✘ for ✔, add license, add br
[bulat.safiullin] [BEAM-14040] [Website] add empty line
[bulat.safiullin] [BEAM-14040] [Website] change td
[bulat.safiullin] [BEAM-14041] [Website] update built io transforms
[bulat.safiullin] [BEAM-14041] [Website] move connectors from Miscellaneous to Database
[bulat.safiullin] [BEAM-14040] [Website] change links color
[danielamartinmtz] Updated metrics' CronJob API to use the latest batch version.
[bulat.safiullin] [BEAM-14041] [Website] change IO from go to java
[bulat.safiullin] [BEAM-14040] [Website] change links, change specific version to current
[danielamartinmtz] Updated cluster to test in metrics-upgrade-clone in BeamMetrics_Publish
[aydar.zaynutdinov] [BEAM-13976][Website]
[aydar.zaynutdinov] [BEAM-14040][Website]
[aydar.zaynutdinov] [BEAM-14041][Website]
[danielamartinmtz] Updated StateFulSet k8s obejct in cassandra-svc-statefulset.yaml file in
[danielamartinmtz] Updated documentation including cluster specs.
[noreply] Beam 13058 k8s apis upgrade - elasticsearch (#18)
[danielamartinmtz] Removed code used for testing.
[danielamartinmtz] Removed code used for testing in job_PostCommit_BeamMetrics_Publish
[noreply] Beam 13058 k8s apis upgrade - Adding Basic Auth details in documentation
[Pablo Estrada] [BEAM-14151] Excluding Spanner CDC tests from Dataflow V1 suite
[danielamartinmtz] Added comments in initContainers and remove unused code in elasticsearch
[noreply] [BEAM-14134] Optimize memory allocations for various core coders
[noreply] [BEAM-14129] Restructure PubsubLiteIO Read side to produce smaller
[noreply] [BEAM-12697] Add primitive field generation from IR to SBE extension
[noreply] [BEAM-13889] Add test cases to jsonx package (#17124)
[noreply] Remove unreachable code in container.go (#17166)
[noreply] Add ability to handle streaming input to AvroSchemaIOProvider (#17126)
[noreply] [BEAM-12898] Flink Load Tests failure- UncheckedExecutionException -
[Daniel Oliveira] Moving to 2.39.0-SNAPSHOT on master branch.
------------------------------------------
[...truncated 321.93 KB...]
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.39.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.25
Using cached botocore-1.24.25-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.39.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.39.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.45.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.39.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.39.0.dev0-py3-none-any.whl size=2695955 sha256=b4bd2e80db9335fbc8dc15f94c038833e7d746638289a7f5168736553eea73da
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.39.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.25 botocore-1.24.25 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.45.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.39.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0908aa58c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0908aa60e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0324100424.1648135659.556589/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0324100424.1648135659.556589/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0324100424.1648135659.556589/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0324100424.1648135659.556589/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220324152739557635-1844'
createTime: '2022-03-24T15:27:40.670330Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-24_08_27_40-6543669790718279820'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0324100424'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-24T15:27:40.670330Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-24_08_27_40-6543669790718279820]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-24_08_27_40-6543669790718279820
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-24_08_27_40-6543669790718279820?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-24_08_27_40-6543669790718279820 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:43.317Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.383Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.451Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.485Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.534Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.569Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.601Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.629Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.664Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.695Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.736Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.776Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.816Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.838Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.875Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:44.911Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:45.103Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:45.176Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:45.217Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:45.243Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-24_08_27_40-6543669790718279820 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:27:58.303Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:28:14.671Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:28:14.703Z: JOB_MESSAGE_DETAILED: Resized **** pool to 5, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:28:24.894Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:28:45.280Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.002Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.071Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.106Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.232Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.282Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.351Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T15:35:16.418Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T20:00:28.756Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-24_08_27_40-6543669790718279820.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T20:00:28.792Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T20:00:28.950Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T20:00:29.083Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-24T20:00:29.108Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-24_08_27_40-6543669790718279820 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 34m 54s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/c2sem72evxqsq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #531
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/531/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13232] Close clients properly in KinesisSource. Also use lazy init
[noreply] [BEAM-14141] Set Interactive Beam to use the default Dataproc image
[noreply] BEAM-14115 - Update find criteria limited to _id (#17102)
[chamikaramj] Disable BigQueryIOStorageWriteIT for general Java post-commit
[noreply] Revert "[BEAM-14038] Auto-startup for Python expansion service.
[noreply] Minor: Bump timeout for Java PreCommit (#17157)
[noreply] [BEAM-14152] Disable flaky
[noreply] Fixing a small bug in TypedSchemaTransformTest that caused it to flake.
[noreply] [BEAM-14116] Catch MonitoringInfoMetricName null keys or values in the
[noreply] [BEAM-14129] Restructure SubscriptionPartitionLoader to use a manual SDF
[noreply] [BEAM-13015] Avoid repeated weighing of StateKey in
[noreply] Add option to add modules to JDK add-open (#17110)
[noreply] [BEAM-13015] Clarify ownership of the list for state caching across
------------------------------------------
[...truncated 323.33 KB...]
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.24
Using cached botocore-1.24.24-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2695956 sha256=f6649dbd195b41c5b30106a2acf8416456ed4f9750af0abdb360a05abd8ddf23
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.24 botocore-1.24.24 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1206a0da70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1206a14290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0323100424.1648051330.965742/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0323100424.1648051330.965742/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0323100424.1648051330.965742/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0323100424.1648051330.965742/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220323160210966689-3325'
createTime: '2022-03-23T16:02:12.129799Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-23_09_02_11-1733959984441234135'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0323100424'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-23T16:02:12.129799Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-23_09_02_11-1733959984441234135]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-23_09_02_11-1733959984441234135
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-23_09_02_11-1733959984441234135?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-23_09_02_11-1733959984441234135 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:14.447Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.691Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.725Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.759Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.791Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.825Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.873Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.900Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.926Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.952Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:15.977Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.014Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.056Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.088Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.124Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.151Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.300Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.381Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.426Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:16.480Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-23_09_02_11-1733959984441234135 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:46.447Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:46.483Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:52.429Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:02:56.669Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:03:17.046Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.571Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.692Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.725Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.787Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.838Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.918Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T16:09:50.999Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T20:00:28.281Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-23_09_02_11-1733959984441234135.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T20:00:28.307Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T20:00:28.463Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T20:00:28.598Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-23T20:00:28.654Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-23_09_02_11-1733959984441234135 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 10s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/66s45cdll7a2g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #530
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/530/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-14124] Add display data to BQ storage reads.
[mmack] [adhoc] Move aws IT tests to testing package according to best practices
[noreply] fixes static checks and go lint issues (#17138)
[Kyle Weaver] Don't print in task configuration.
[noreply] [BEAM-14136] Clean up staticcheck and linter warnings in the Go SDK
[noreply] Merge pull request #17063 from [BEAM-12164] Fix flaky tests
[noreply] Revert "[BEAM-14112] Avoid storing a generator in _CustomBigQuerySource
[Kyle Weaver] [BEAM-4106] Remove filesToStage from Flink pipeline option list.
[noreply] [BEAM-14071] Enabling Flink on Dataproc for Interactive Beam (#17044)
[noreply] Minor: Bypass schema registry in schemas_test.py (#17108)
------------------------------------------
[...truncated 322.16 KB...]
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.23
Using cached botocore-1.24.23-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2695811 sha256=3c318d1278ddb1f2f48c2ae36e5f06731118195ebc4a2327423fed42c1482683
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.23 botocore-1.24.23 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f336fca3a70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f336fca9290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0322100403.1647965181.925019/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0322100403.1647965181.925019/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0322100403.1647965181.925019/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0322100403.1647965181.925019/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220322160621925923-7816'
createTime: '2022-03-22T16:06:22.965893Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-22_09_06_22-1518280807633814000'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0322100403'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-22T16:06:22.965893Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-22_09_06_22-1518280807633814000]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-22_09_06_22-1518280807633814000
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-22_09_06_22-1518280807633814000?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-22_09_06_22-1518280807633814000 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:26.620Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.102Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.139Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.189Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.223Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.254Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.283Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.318Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.352Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.396Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.433Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.465Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.502Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.542Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.574Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:28.622Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:29.198Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:29.290Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:29.336Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:29.395Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-22_09_06_22-1518280807633814000 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:06:46.544Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:07:08.363Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:07:30.810Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:50.939Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.002Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.025Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.091Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.159Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.232Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T16:13:51.300Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T20:00:38.932Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-22_09_06_22-1518280807633814000.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T20:00:38.959Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T20:00:39.122Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T20:00:39.197Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-22T20:00:39.232Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-22_09_06_22-1518280807633814000 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 56m 13s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/axqmcbfffqhce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #529
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/529/display/redirect>
Changes:
------------------------------------------
[...truncated 323.62 KB...]
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.22
Using cached botocore-1.24.22-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694808 sha256=1e22346daae543fbba8eae8bb32a551175754a6b97e195c3384b81bb8f4653b7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.22 botocore-1.24.22 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f90717d6560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f90717d6d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0321100420.1647878827.980490/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0321100420.1647878827.980490/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0321100420.1647878827.980490/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0321100420.1647878827.980490/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220321160707981428-9182'
createTime: '2022-03-21T16:07:09.647319Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-21_09_07_08-1746181686287769854'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0321100420'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-21T16:07:09.647319Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-21_09_07_08-1746181686287769854]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-21_09_07_08-1746181686287769854
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-21_09_07_08-1746181686287769854?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-21_09_07_08-1746181686287769854 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:17.810Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.704Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.732Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.791Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.817Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.851Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.874Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.922Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.957Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:18.979Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.001Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.033Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.058Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.079Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.203Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.226Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.354Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.409Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.447Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:19.481Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-21_09_07_08-1746181686287769854 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:48.736Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:54.662Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:07:54.690Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:08:04.951Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:08:28.284Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.133Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.204Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.237Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.303Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.359Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.422Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T16:14:56.490Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T20:00:25.755Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-21_09_07_08-1746181686287769854.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T20:00:25.783Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T20:00:26.008Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T20:00:26.947Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-21T20:00:26.974Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-21_09_07_08-1746181686287769854 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 55m 12s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/d4qdxgof6saa6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #528
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/528/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14122] Upgrade pip-licenses dependency (#17132)
------------------------------------------
[...truncated 321.75 KB...]
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting botocore<1.25.0,>=1.24.22
Using cached botocore-1.24.22-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694808 sha256=24a06bf89d413e88cec9b9b53f74719df5a05881651df69ce31ef589fb9f5fbd
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.22 botocore-1.24.22 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fda4def2560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fda4def2d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0320100400.1647792343.909559/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0320100400.1647792343.909559/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0320100400.1647792343.909559/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0320100400.1647792343.909559/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220320160543910602-4714'
createTime: '2022-03-20T16:05:44.903506Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-20_09_05_44-16280827193404219038'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0320100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-20T16:05:44.903506Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-20_09_05_44-16280827193404219038]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-20_09_05_44-16280827193404219038
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-20_09_05_44-16280827193404219038?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-20_09_05_44-16280827193404219038 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:47.424Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.291Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.332Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.362Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.395Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.438Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.491Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.514Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.540Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.563Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.595Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.624Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.660Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.687Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.720Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.742Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.879Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.928Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.965Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-20_09_05_44-16280827193404219038 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:05:49.994Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:06:24.523Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:06:29.929Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:06:57.469Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:06:57.508Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.541Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.613Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.648Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.714Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.782Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.835Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T16:13:14.904Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T20:00:39.366Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-20_09_05_44-16280827193404219038.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T20:00:39.405Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T20:00:39.601Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T20:00:39.670Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-20T20:00:39.690Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-20_09_05_44-16280827193404219038 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 56m 47s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/64nqd33l6jwpm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #527
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/527/display/redirect?page=changes>
Changes:
[Kiley Sok] Add Java 17 Nexmark metrics to Grafana
[yiru] .
[yiru] .
[yiru] .
[yiru] format fix
[yiru] .
[yiru] make DoFn into a separate class
[yiru] .
[yiru] fix setting
[mmack] [adhoc] Minor cleanup for aws2 tests
[mmack] [BEAM-14125] Update website IO matrix to recommend aws2 IOs
[noreply] [BEAM-14128] Eliminating quadratic behavior of
[noreply] [BEAM-13972] Add RunInference interface (#16917)
[noreply] Merge pull request #17116 from [BEAM-12164] Remove change_stream in
[yiru] fix checkstyle
[yiru] spotlessapply
[noreply] Deprecate tags.go (#17025)
[noreply] [BEAM-12753] and [BEAM-12815] Fix Flink Integration Tests (#17067)
[noreply] Merge pull request #16895 from [BEAM-13882][Playground] More tests for
[noreply] [BEAM-13925] Add weekly automation to update our reviewer config
[noreply] Merge pull request #17076 from Beam 14082 update payground for mobile
[noreply] [BEAM-13925] Assign committers in the scheduled action (#17062)
[noreply] Pin setup-gcloud to v0 instead of master (#17123)
[noreply] [BEAM-3304] documentation for PaneInfo in BPG (#17047)
[noreply] Merge pull request #17016 from [BEAM-14049][Playground] Add new API
[noreply] Merge pull request #17077 from [BEAM-14078] [Website] change link
[noreply] Merge pull request #17085 from [BEAM-14077] [Website] add beam
[noreply] Update Changes.md w/Go pipeline pre-process fix.
[noreply] [BEAM-14098] wrapper for postgres on JDBC IO GO SDK (#17088)
[noreply] Merge pull request #17023 from [BEAM-12164]: Remove child partition
------------------------------------------
[...truncated 321.42 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.22-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.22
Using cached botocore-1.24.22-py3-none-any.whl (8.6 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694808 sha256=b53c307e702f9dc81b5e67ec54a52da4a585eef8d803c707e4bc2a836efaad7b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.22 botocore-1.24.22 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1112e0b560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1112e0bd40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0319100417.1647704235.000954/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0319100417.1647704235.000954/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0319100417.1647704235.000954/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0319100417.1647704235.000954/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220319153715001856-8132'
createTime: '2022-03-19T15:37:15.993856Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-19_08_37_15-12411302383755379330'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0319100417'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-19T15:37:15.993856Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-19_08_37_15-12411302383755379330]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-19_08_37_15-12411302383755379330
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-19_08_37_15-12411302383755379330?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-19_08_37_15-12411302383755379330 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.098Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.764Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.807Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.844Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.876Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.916Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.950Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:19.982Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.018Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.077Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.108Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.177Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.208Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.239Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.289Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.418Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.494Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.529Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:20.560Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-19_08_37_15-12411302383755379330 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:32.821Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:49.631Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:49.674Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:37:59.867Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:38:20.789Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:38:20.824Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:14.978Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.038Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.066Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.132Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.182Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.247Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T15:44:15.319Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T20:00:35.734Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-19_08_37_15-12411302383755379330.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T20:00:35.761Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T20:00:35.965Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T20:00:36.092Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-19T20:00:36.115Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-19_08_37_15-12411302383755379330 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 25m 9s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/uvu2ftavj2dpw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #526
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/526/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-10212] Clean-up comments, remove rawtypes usage.
[noreply] [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner
[noreply] [BEAM-12777] Create symlink for `current` directory (#17105)
[noreply] [BEAM-14020] Adding SchemaTransform, SchemaTransformProvider,
[noreply] [BEAM-13015] Modify metrics to begin and reset to a non-dirty state.
[noreply] [BEAM-14112] Avoid storing a generator in _CustomBigQuerySource (#17100)
[noreply] Populate environment capabilities in v1beta3 protos. (#17042)
[Kyle Weaver] [BEAM-12976] Test a whole pipeline using projection pushdown in BQ IO.
[Kyle Weaver] [BEAM-12976] Enable projection pushdown for Java pipelines on Dataflow,
[noreply] [BEAM-14038] Auto-startup for Python expansion service. (#17035)
[Kyle Weaver] [BEAM-14123] Fix typo in hdfsIntegrationTest task name.
[noreply] [BEAM-13893] improved coverage of jobopts package (#17003)
[noreply] Merge pull request #16977 from [BEAM-12164] Added integration test for
------------------------------------------
[...truncated 321.70 KB...]
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.21
Using cached botocore-1.24.21-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.0-py2.py3-none-any.whl (241 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694808 sha256=83bb87d04d9e005cb8a4ee0aec71d84d43a376d0389cdcd70d956d3825c97a7f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.21 botocore-1.24.21 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.56.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f63886f6560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f63886f6d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0318100411.1647619846.001359/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0318100411.1647619846.001359/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0318100411.1647619846.001359/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0318100411.1647619846.001359/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220318161046002294-3073'
createTime: '2022-03-18T16:10:47.044876Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-18_09_10_46-11153347750513266615'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0318100411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-18T16:10:47.044876Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-18_09_10_46-11153347750513266615]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-18_09_10_46-11153347750513266615
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-18_09_10_46-11153347750513266615?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-18_09_10_46-11153347750513266615 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:50.282Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:51.784Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:51.850Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:51.950Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.036Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.118Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.146Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.180Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.215Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.271Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.302Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.337Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.379Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.418Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.452Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.487Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.669Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.726Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.772Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:10:52.793Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-18_09_10_46-11153347750513266615 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:25.030Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:25.814Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:25.847Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:36.060Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:59.779Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:11:59.812Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:32.912Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:32.985Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:33.026Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:33.086Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:33.135Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:33.209Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T16:18:33.269Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T20:00:25.917Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-18_09_10_46-11153347750513266615.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T20:00:25.970Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T20:00:26.141Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T20:00:26.255Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-18T20:00:26.348Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-18_09_10_46-11153347750513266615 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 51m 33s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/e7y5lsiqgmhx2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #525
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/525/display/redirect?page=changes>
Changes:
[noreply] Mapped JOB_STATE_RESOURCE_CLEANING_UP to State.RUNNING.
[ryanthompson591] fixed typo in typehints
[zyichi] Remove unused prebuild_sdk_container_base_iamge option from validate
[hengfeng] feat: add more custom metrics
[noreply] [BEAM-14103][Playgrounf][Bugfix] Fix google analytics id (#17092)
[noreply] Minor: Make ScopedReadStateSupplier final (#16992)
[noreply] [BEAM-14113] Improve SamzaJobServerDriver extensibility (#17099)
[noreply] [BEAM-14116] Chunk commit requests dynamically (#17004)
[noreply] Merge pull request #17079 from [BEAM-13660] Add types and queries in
[noreply] [BEAM-13888] Add unit testing to ioutilx (#17058)
[noreply] Merge pull request #16822 from [BEAM-13841][Playground] Add Application
[noreply] Minor: Make serializableCoder warning gramatically correct english
[noreply] [BEAM-14091] Fixing Interactive Beam show/collect for remote runners
------------------------------------------
[...truncated 323.11 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.21-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.21
Using cached botocore-1.24.21-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.10.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694797 sha256=6cd024d55c14a87838f9bd9f8f81fc8fe639aa809da53377c130b35b0f735308
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.21 botocore-1.24.21 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-1.0.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f65de69a560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f65de69ad40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0317100410.1647533563.746554/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0317100410.1647533563.746554/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0317100410.1647533563.746554/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0317100410.1647533563.746554/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220317161243747464-5803'
createTime: '2022-03-17T16:12:44.790968Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-17_09_12_44-1865927407998196881'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0317100410'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-17T16:12:44.790968Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-17_09_12_44-1865927407998196881]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-17_09_12_44-1865927407998196881
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-17_09_12_44-1865927407998196881?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-17_09_12_44-1865927407998196881 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:06.534Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.399Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.457Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.500Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.548Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.646Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.690Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.743Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.782Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.827Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.857Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.890Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.929Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.960Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:07.993Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:08.028Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:08.164Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:08.229Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:08.284Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:08.327Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-17_09_12_44-1865927407998196881 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:37.809Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:37.851Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:41.291Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:13:48.044Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:14:12.009Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:14:12.033Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.244Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.314Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.389Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.517Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.581Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.643Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T16:20:33.698Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T20:00:34.885Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-17_09_12_44-1865927407998196881.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T20:00:34.995Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T20:00:35.195Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-17_09_12_44-1865927407998196881 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T20:00:41.603Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-17T20:00:41.667Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 49m 46s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/me5wqeeontmv4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #524
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/524/display/redirect?page=changes>
Changes:
[Chamikara Madhusanka Jayalath] Updates x-lang release validation to use staged jars
[dhuntsperger] documented maven-to-gradle conversion for Dataflow; refactored java
[dhuntsperger] adding a list of example pipelines
[dhuntsperger] removing unnecessary `ls` command from maven project generation
[dhuntsperger] fixing filename formatting in response to feedback
[dhuntsperger] adding extra step emphasizing runner setupt
[dhuntsperger] reorganized instructions to emphasize setup steps for runners
[noreply] [BEAM-13767] Move a bunch of python tasks to use gradle configuration…
[noreply] Merge pull request #17052 from [BEAM-13818] [SnowflakeIO] Add support
[noreply] Adding pydoc for StateHandler (#17091)
[noreply] BEAM-3165 Bypass split if numSplit is zero (#17084)
------------------------------------------
[...truncated 322.16 KB...]
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.20
Using cached botocore-1.24.20-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694465 sha256=60a89c665c2c1b6224b422953326844649bf4964e8ef2af75625e944b8c3073b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.20 botocore-1.24.20 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.2 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.9 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f02447387a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0244738f80> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0316100418.1647446125.910081/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0316100418.1647446125.910081/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0316100418.1647446125.910081/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0316100418.1647446125.910081/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220316155525911011-9985'
createTime: '2022-03-16T15:55:26.976541Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-16_08_55_26-12672782891714741740'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0316100418'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-16T15:55:26.976541Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-16_08_55_26-12672782891714741740]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-16_08_55_26-12672782891714741740
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-16_08_55_26-12672782891714741740?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-16_08_55_26-12672782891714741740 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:29.090Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.124Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.161Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.189Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.232Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.274Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.301Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.335Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.356Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.387Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.421Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.446Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.479Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.570Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.624Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.655Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.784Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.840Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.876Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:30.916Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-16_08_55_26-12672782891714741740 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:55:59.934Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:56:01.367Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 1 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:56:01.390Z: JOB_MESSAGE_DETAILED: Resized **** pool to 1, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:56:11.649Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:56:32.861Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T15:56:32.889Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:05.820Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:05.912Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:05.936Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:06.017Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:06.066Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:06.128Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T16:03:06.183Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T20:00:28.888Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-16_08_55_26-12672782891714741740.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T20:00:28.917Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T20:00:29.805Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-16_08_55_26-12672782891714741740 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T20:00:31.987Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-16T20:00:32.011Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 54s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/zvkbus5nwp3us
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #523
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/523/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-12572] Add DistinctExample test
[benjamin.gonzalez] [BEAM-12572] Add DistinctExample test
[benjamin.gonzalez] [BEAM-12572] Add java examples tests
[benjamin.gonzalez] [BEAM-12572] Change tests collected by runner
[benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions, skip tests for specific runners
[benjamin.gonzalez] [BEAM-12572] Skip failing tests
[benjamin.gonzalez] [BEAM-12572] Fix checkstyle warnings
[benjamin.gonzalez] [BEAM-12572] Skip tfidf tests on direct runner
[benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions
[benjamin.gonzalez] [BEAM-12572] Fix checkstyle warning
[benjamin.gonzalez] [BEAM-12572] Enable test with Pipeline serialization error fixed, link
[benjamin.gonzalez] [BEAM-12572] Sickbay TfIdf test for DirectRunner
[benjamin.gonzalez] [BEAM-12572] Refactor naming for BigQueryClient
[mmack] [BEAM-13175] Add KinesisIO.write for AWS SDK v2.
[mmack] [adhoc] Update owners for aws modules
[noreply] [BEAM-14051] Add a warning to gradle.properties that many Beam modules
[noreply] Merge pull request #17000 from [BEAM-13881] [Playground] Increase test
[noreply] Add deterministic dict coding via key sorting. (#17037)
[noreply] Merge pull request #17054 from [BEAM-14075] [SnowflakeIO] Change a
[mmack] [BEAM-13175] Document changes to IOs in amazon-web-services2 for AWS
------------------------------------------
[...truncated 321.75 KB...]
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.19-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.19
Using cached botocore-1.24.19-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2694174 sha256=fb23bf1968cf17f5097202b25b06912952fd2e4f9eab70e5ee51657cd18fe874
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.19 botocore-1.24.19 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff0358e77a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff0358e7f80> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0315100420.1647360271.803756/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0315100420.1647360271.803756/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0315100420.1647360271.803756/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0315100420.1647360271.803756/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220315160431804681-3502'
createTime: '2022-03-15T16:04:32.876350Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-15_09_04_32-3838706763420577245'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0315100420'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-15T16:04:32.876350Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-15_09_04_32-3838706763420577245]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-15_09_04_32-3838706763420577245
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-15_09_04_32-3838706763420577245?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-15_09_04_32-3838706763420577245 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:35.532Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.734Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.766Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.804Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.836Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.889Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.938Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.974Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:36.998Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.034Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.067Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.101Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.135Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.167Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.201Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.224Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.373Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.429Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.482Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:37.510Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-15_09_04_32-3838706763420577245 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:04:45.480Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:05:05.782Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:05:05.820Z: JOB_MESSAGE_DETAILED: Resized **** pool to 5, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:05:16.057Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:05:39.094Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:05:39.128Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.573Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.652Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.694Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.787Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.853Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:17.935Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T16:12:18Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T20:01:04.287Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-15_09_04_32-3838706763420577245.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T20:01:04.325Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T20:01:04.503Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T20:01:04.643Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-15T20:01:04.667Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-15_09_04_32-3838706763420577245 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 58m 26s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/4bozcjiufxf2o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #522
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/522/display/redirect>
Changes:
------------------------------------------
[...truncated 324.63 KB...]
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.2-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.18
Using cached botocore-1.24.18-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693741 sha256=67df639b4c06e5f3ab5434f0fe166e1c56b2d3bbee7c89e95c26fbd256e8b8b8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.18 botocore-1.24.18 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0f8a2790e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0f8a2798c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0314100358.1647273933.097660/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0314100358.1647273933.097660/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0314100358.1647273933.097660/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0314100358.1647273933.097660/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220314160533098630-3128'
createTime: '2022-03-14T16:05:34.087233Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-14_09_05_33-6809173653286623505'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0314100358'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-14T16:05:34.087233Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-14_09_05_33-6809173653286623505]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-14_09_05_33-6809173653286623505
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-14_09_05_33-6809173653286623505?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-14_09_05_33-6809173653286623505 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:45.062Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.477Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.529Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.565Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.601Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.655Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.695Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.734Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.779Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.824Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.849Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.886Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.921Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.953Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:47.985Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:48.014Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:48.245Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:48.330Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:48.393Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:05:48.435Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-14_09_05_33-6809173653286623505 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:06:13.615Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:06:33.760Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:07:00.078Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:07:00.112Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.363Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.498Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.556Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.634Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.721Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.804Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T16:13:25.908Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T20:00:44.664Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-14_09_05_33-6809173653286623505.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T20:00:44.736Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T20:00:44.978Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-14_09_05_33-6809173653286623505 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T20:00:49.767Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-14T20:00:49.813Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 57m 10s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/7d7dtpsr67wn6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #521
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/521/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14072] [BEAM-13993] [BEAM-10039] Import beam plugins before
------------------------------------------
[...truncated 322.80 KB...]
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.18
Using cached botocore-1.24.18-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693741 sha256=5f61916125b37a10320f4f3b8e3ebdc32c4ea07b4631c1a357b0b375ccfa47e6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.18 botocore-1.24.18 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f46295d00e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f46295d08c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0313100411.1647187469.035380/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0313100411.1647187469.035380/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0313100411.1647187469.035380/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0313100411.1647187469.035380/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220313160429036321-7897'
createTime: '2022-03-13T16:04:30.101580Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-13_09_04_29-1824346931256155339'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0313100411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-13T16:04:30.101580Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-13_09_04_29-1824346931256155339]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-13_09_04_29-1824346931256155339
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-13_09_04_29-1824346931256155339?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-13_09_04_29-1824346931256155339 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:32.703Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.743Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.783Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.819Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.855Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.893Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.932Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.963Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:33.995Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.026Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.059Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.093Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.128Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.159Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.192Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.225Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.364Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.435Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.482Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:34.511Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-13_09_04_29-1824346931256155339 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:04:58.130Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:05:06.126Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 7 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:05:06.155Z: JOB_MESSAGE_DETAILED: Resized **** pool to 7, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:05:16.367Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:05:39.022Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:05:39.056Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.046Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.157Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.190Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.245Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.337Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.401Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T16:12:06.458Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T20:00:37.042Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-13_09_04_29-1824346931256155339.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T20:00:37.074Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T20:00:37.233Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-13_09_04_29-1824346931256155339 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T20:00:39.292Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-13T20:00:39.313Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 57m 56s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/xfc63imcghcta
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #520
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/520/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-13981] Remove Spark Runner specific code for event logging
[vitaly.terentyev] [BEAM-2766] Support null key/values in HadoopFormatIO
[vitaly.terentyev] [BEAM-2766] Fix checkstyle
------------------------------------------
[...truncated 322.69 KB...]
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.18
Using cached botocore-1.24.18-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693483 sha256=15912e1da49a84615b68e9721bc126d7318eac909103d8eb3f9e7f8a20f5fddc
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.18 botocore-1.24.18 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1e77c680e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1e77c688c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0312100355.1647100438.546704/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0312100355.1647100438.546704/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0312100355.1647100438.546704/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0312100355.1647100438.546704/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220312155358547634-6757'
createTime: '2022-03-12T15:53:59.753387Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-12_07_53_59-5569589886325518293'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0312100355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-12T15:53:59.753387Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-12_07_53_59-5569589886325518293]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-12_07_53_59-5569589886325518293
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-12_07_53_59-5569589886325518293?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-12_07_53_59-5569589886325518293 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:02.444Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.205Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.249Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.275Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.306Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.352Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.389Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.426Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.461Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.520Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.552Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.585Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.624Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.659Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.680Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.715Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.857Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.923Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.966Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:05.985Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-12_07_53_59-5569589886325518293 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:40.840Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:41.115Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:41.145Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:54:51.389Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:55:14.237Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T15:55:14.266Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:30.193Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:30.613Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:30.720Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:30.852Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:30.929Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:31.001Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T16:01:31.076Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T20:00:34.175Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-12_07_53_59-5569589886325518293.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T20:00:34.224Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T20:00:34.791Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T20:00:35.038Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-12T20:00:35.065Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-12_07_53_59-5569589886325518293 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 8m 26s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/dntcpgu5hwxbw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #519
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/519/display/redirect?page=changes>
Changes:
[ihr] [BEAM-13923] Fix the answers placeholders locations in the Java katas
[jakub.kukul] [BEAM-14039] Propagate ignore_unknown_columns parameter.
[stranniknm] [BEAM-14079] playground - improve accessibility
[noreply] [BEAM-13925] Find and address prs that havent been reviewed in a week
[noreply] Fix import path
[noreply] [BEAM-13925] Fix one more import path
[noreply] Add a StatefulDoFn test that sets event time timer within allowed
[noreply] Merge pull request #17056 from [BEAM-14076] [SnowflakeIO] Add support
------------------------------------------
[...truncated 322.08 KB...]
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.17-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.17
Using cached botocore-1.24.17-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693483 sha256=28a1e293f54b2652b1867ab354cddd6f9c3b85a1bc6cfd864ea2f2f91874f7ef
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.17 botocore-1.24.17 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0bab0dc0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0bab0dc8c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0311100353.1647013495.606006/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0311100353.1647013495.606006/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0311100353.1647013495.606006/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0311100353.1647013495.606006/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220311154455606931-6472'
createTime: '2022-03-11T15:44:56.595531Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-11_07_44_56-15611626937673886861'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0311100353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-11T15:44:56.595531Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-11_07_44_56-15611626937673886861]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-11_07_44_56-15611626937673886861
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-11_07_44_56-15611626937673886861?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-11_07_44_56-15611626937673886861 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:58.642Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.443Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.483Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.543Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.577Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.620Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.655Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.687Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.721Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.759Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.790Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.825Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.848Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.879Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.903Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:44:59.927Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:00.114Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:00.182Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:00.227Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:00.249Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-11_07_44_56-15611626937673886861 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:31.536Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:36.719Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:36.771Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:45:47.094Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:46:07.258Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.153Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.224Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.280Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.323Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.360Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.432Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T15:52:33.509Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T20:00:35.215Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-11_07_44_56-15611626937673886861.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T20:00:35.247Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T20:00:35.342Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-11_07_44_56-15611626937673886861 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T20:00:40.953Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-11T20:00:40.978Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 17m 33s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/y4jtzbog53ryg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #518
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/518/display/redirect?page=changes>
Changes:
[hengfeng] [BEAM-12164]: display the metadata table's name on UI
[noreply] Revert "[BEAM-13993] [BEAM-10039] Import beam plugins before starting
[noreply] Merge pull request #17036 from [BEAM-12164] Convert all static instances
[noreply] fix variable reference (#16991)
[noreply] Merge pull request #16844 from [BEAM-12164]: allow for nanosecond
[noreply] [BEAM-13904] Increase unit testing in the reflectx package (#17024)
------------------------------------------
[...truncated 321.97 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.16-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.10.0-py3-none-any.whl (346 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.23.0-py3-none-any.whl (179 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.16
Using cached botocore-1.24.16-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693407 sha256=353abbd8f00554dd428ff6d048148b89011af96dc0893e634d496fb67a35c754
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.10.0 boto3-1.21.16 botocore-1.24.16 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.13.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.11.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.14.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff04ea6d0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff04ea6d8c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646928248.521853/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646928248.521853/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646928248.521853/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646928248.521853/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220310160408522789-7036'
createTime: '2022-03-10T16:04:09.563898Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-10_08_04_09-8830839273733079577'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0308201426'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-10T16:04:09.563898Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-10_08_04_09-8830839273733079577]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-10_08_04_09-8830839273733079577
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-10_08_04_09-8830839273733079577?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-10_08_04_09-8830839273733079577 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:13.474Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.559Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.633Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.667Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.701Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.764Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.811Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.847Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.880Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.918Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:19.970Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.003Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.045Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.071Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.114Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.150Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.312Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.383Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.451Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:20.485Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-10_08_04_09-8830839273733079577 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:04:40.632Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:05:11.224Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:05:33.146Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:05:33.180Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.525Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.660Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.699Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.810Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.873Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:02.972Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T16:12:03.074Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-10_08_04_09-8830839273733079577 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T20:00:41.383Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-10_08_04_09-8830839273733079577.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T20:00:41.419Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T20:00:41.623Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T20:00:47.628Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-10T20:00:47.675Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 58m 20s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/4duv4ba23w7ze
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #517
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/517/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Update dataflow API client.
[Robert Bradshaw] Instructions for updating apitools generated files.
[noreply] Merge pull request #17027: [BEAM-11205] Upgrade GCP Libraries BOM
[noreply] [BEAM-13709] Inconsistent behavior when parsing boolean flags across
[noreply] [BEAM-10976] Bundle finalization: Harness and some exec changes (#16980)
[noreply] Merge pull request #16976 from [BEAM-14010] [Website] Add Playground
[noreply] [BEAM-12447] Upgrade cloud build client and add/cleanup options (#17032)
------------------------------------------
[...truncated 321.22 KB...]
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.15
Using cached botocore-1.24.15-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2693718 sha256=24afdfe3bde8694de83c0c3532fac00c2d40990988cad61e0bafbeead106a3d2
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.15 botocore-1.24.15 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.10.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.2 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7f47675f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7f476767a0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646838723.979620/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646838723.979620/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646838723.979620/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308201426.1646838723.979620/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220309151203980709-2215'
createTime: '2022-03-09T15:12:05.140701Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-09_07_12_04-4096272222805556041'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0308201426'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-09T15:12:05.140701Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-09_07_12_04-4096272222805556041]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-09_07_12_04-4096272222805556041
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-09_07_12_04-4096272222805556041?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-09_07_12_04-4096272222805556041 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:07.568Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.102Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.138Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.179Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.226Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.263Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.289Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.323Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.358Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.380Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.405Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.466Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.495Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.535Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.568Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.697Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.746Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.797Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:10.828Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-09_07_12_04-4096272222805556041 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:32.848Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:52.049Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 6 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:12:52.068Z: JOB_MESSAGE_DETAILED: Resized **** pool to 6, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:13:02.298Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:13:21.987Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:13:22.027Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:16.683Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:16.800Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:16.836Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:16.902Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:16.946Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:17.007Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T15:19:17.070Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T20:00:56.922Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-09_07_12_04-4096272222805556041.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T20:00:56.951Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T20:00:57.218Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T20:00:57.738Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-09T20:00:57.762Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-09_07_12_04-4096272222805556041 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 50m 46s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/s4s5nsc5v7izc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #516
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/516/display/redirect?page=changes>
Changes:
[dannymccormick] [BEAM-11085] Test that windows are correctly observed in DoFns
[jrmccluskey] [BEAM-14050] Update taxi.go example instructions
[noreply] Give pr bot write permissions on pr update
[noreply] Adding a logical type for Schemas using proto serialization. (#16940)
[noreply] BEAM-13765 missing PAssert methods (#16668)
[noreply] [BEAM-13909] improve coverage of Provision package (#17014)
------------------------------------------
[...truncated 322.65 KB...]
Using cached SQLAlchemy-1.4.32-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.14-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.14
Using cached botocore-1.24.14-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.1-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2680293 sha256=6ca581565b0c125b4c63fbb01f02fb469fe4b406ab7333c9d9944e720e8e9d65
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.14 botocore-1.24.14 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.2 google-cloud-language-1.3.0 google-cloud-pubsub-2.10.0 google-cloud-pubsublite-1.4.1 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.1 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0fbd610290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0fbd610a70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308101242.1646752264.253754/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308101242.1646752264.253754/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308101242.1646752264.253754/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0308101242.1646752264.253754/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220308151104254863-2416'
createTime: '2022-03-08T15:11:05.457621Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-08_07_11_04-6277360990853343816'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0308101242'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-08T15:11:05.457621Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-08_07_11_04-6277360990853343816]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-08_07_11_04-6277360990853343816
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-08_07_11_04-6277360990853343816?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-08_07_11_04-6277360990853343816 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:08.797Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.688Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.745Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.781Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.836Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.914Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.946Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:09.977Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.033Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.068Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.100Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.132Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.181Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.213Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.243Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.295Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.513Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-08_07_11_04-6277360990853343816 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.599Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.647Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:10.705Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:27.752Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:11:57.812Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:12:20.334Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:12:20.358Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:36.655Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:36.790Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:36.811Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:36.891Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:36.959Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:37.050Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T15:18:37.112Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T20:00:35.948Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-08_07_11_04-6277360990853343816.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T20:00:36.040Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T20:00:36.252Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T20:00:36.724Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-08T20:00:36.754Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-08_07_11_04-6277360990853343816 is in state JOB_STATE_CANCELLING
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 51m 24s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/imo37d3fujcjs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #515
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/515/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13925] Add ability to get metrics on pr-bot performance (#16985)
------------------------------------------
[...truncated 322.41 KB...]
Using cached google_cloud_dlp-3.6.1-py2.py3-none-any.whl (113 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.13
Using cached botocore-1.24.13-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.1-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2680293 sha256=9e44357db99b467c4a47a834c0840572a4738f73bcfabedabd6ff81e9c4aa51d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.13 botocore-1.24.13 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.2 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.10.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.1 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.32 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f674a3e8290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f674a3e8a70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0307101316.1646665898.632238/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0307101316.1646665898.632238/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0307101316.1646665898.632238/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0307101316.1646665898.632238/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220307151138633320-5284'
createTime: '2022-03-07T15:11:39.849980Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-07_07_11_39-13035671848603810788'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0307101316'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-07T15:11:39.849980Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-07_07_11_39-13035671848603810788]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-07_07_11_39-13035671848603810788
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-07_07_11_39-13035671848603810788?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-07_07_11_39-13035671848603810788 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:43.811Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.012Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.056Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.080Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.108Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.177Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.213Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.237Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.275Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.308Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.331Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.358Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.394Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.424Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.457Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.490Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.622Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.693Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.730Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:11:45.761Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-07_07_11_39-13035671848603810788 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:04.462Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:22.391Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 1 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:22.417Z: JOB_MESSAGE_DETAILED: Resized **** pool to 1, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:32.816Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:54.764Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:12:54.787Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.389Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.507Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.546Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.617Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.664Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.716Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T15:19:17.776Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T20:00:24.301Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-07_07_11_39-13035671848603810788.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T20:00:24.330Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T20:00:24.618Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-07_07_11_39-13035671848603810788 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T20:00:30.973Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T20:00:30.995Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 50m 43s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/tbpjm2lk4b2xo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #514
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/514/display/redirect>
Changes:
------------------------------------------
[...truncated 323.16 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.13
Using cached botocore-1.24.13-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.1-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2680293 sha256=8fa414641ada63bf042de705e1e002e65b72c8a0c0af3cfdfb6f52578c6af28d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.13 botocore-1.24.13 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.1 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.10.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.1 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa73ae21290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa73ae21a70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0306143756.1646593197.905869/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0306143756.1646593197.905869/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0306143756.1646593197.905869/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0306143756.1646593197.905869/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220306185957906943-9461'
createTime: '2022-03-06T18:59:59.069713Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-06_10_59_58-18092550872159798640'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0306143756'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-06T18:59:59.069713Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-06_10_59_58-18092550872159798640]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-06_10_59_58-18092550872159798640
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-06_10_59_58-18092550872159798640?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-06_10_59_58-18092550872159798640 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:02.177Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.532Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.571Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.602Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.633Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.671Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.701Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.725Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.762Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.795Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.819Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.847Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.889Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.915Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.961Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:03.987Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:04.112Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-06_10_59_58-18092550872159798640 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:04.174Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:04.216Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:04.250Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:38.428Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:00:51.877Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:01:14.649Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:01:14.673Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:30.669Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:30.837Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:30.882Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:31.030Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:31.114Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:31.178Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-06T19:07:31.254Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T00:00:40.084Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-06_10_59_58-18092550872159798640.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T00:00:40.105Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T00:00:40.227Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T00:00:40.714Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-07T00:00:40.731Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-06_10_59_58-18092550872159798640 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 2m 34s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/s7o3whldwordc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #513
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/513/display/redirect?page=changes>
Changes:
[rahuliyer573] py: Import beam plugins before starting SdkHarness
[stephen.patel] BEAM-14011 fix s3 filesystem multipart copy
[Valentyn Tymofieiev] Bump numpy bound to include 1.22 and regenerate container deps.
[github-actions] [BEAM-13925] months in date constructor are 0 indexed
[noreply] Merge pull request #16842 from [BEAM-13932][Playground] Container's user
[noreply] Doc updates and blog post for 2.37.0 (#16887)
[noreply] Remove resolved issue in docs + update class path on sample (#17018)
[noreply] [BEAM-14016] Fixed flaky postcommit test (#17009)
[noreply] Remove resolved issue in notebook
[noreply] [BEAM-13947] Add split() and rsplit(), non-deferred column operations on
[noreply] BEAM-14026 - Fixes bug related to Unnesting nested rows in an array
------------------------------------------
[...truncated 322.86 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.13
Using cached botocore-1.24.13-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.1-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2680293 sha256=f642e8f623026e74941aaea08b3e89ba5df0a59e0835ac86f1e6648f85d5595a
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.13 botocore-1.24.13 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.10 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.1 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.10.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.1 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff7262a5290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff7262a5a70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0305100401.1646492483.759095/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0305100401.1646492483.759095/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0305100401.1646492483.759095/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0305100401.1646492483.759095/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220305150123760201-6662'
createTime: '2022-03-05T15:01:24.785864Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-05_07_01_24-3934358351742018422'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0305100401'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-05T15:01:24.785864Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-05_07_01_24-3934358351742018422]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-05_07_01_24-3934358351742018422
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-05_07_01_24-3934358351742018422?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-05_07_01_24-3934358351742018422 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:27.402Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.227Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.263Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.288Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.319Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.468Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.553Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.647Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.678Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.702Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.741Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.773Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.802Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.837Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.871Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:28.903Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:29.034Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:29.100Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:29.144Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:29.165Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-05_07_01_24-3934358351742018422 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:01:52.308Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:02:05.235Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 2 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:02:05.269Z: JOB_MESSAGE_DETAILED: Resized **** pool to 2, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:02:15.668Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:02:37.122Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:02:37.149Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.216Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.332Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.382Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.585Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.680Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.762Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T15:08:51.830Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T20:00:34.659Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-05_07_01_24-3934358351742018422.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T20:00:34.689Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T20:00:34.832Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T20:00:34.927Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-05T20:00:34.961Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-05_07_01_24-3934358351742018422 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 59s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/bhkbyyjrb6zra
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #512
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/512/display/redirect?page=changes>
Changes:
[stranniknm] [BEAM-13999] playground - support vertical orientation for graph
[noreply] [adhoc] Prepare aws2 ClientConfiguration for json serialization and
[noreply] Merge pull request #16879 from [BEAM-12164] Add javadocs to
[noreply] [Cleanup] Update pre-v2 go package references (#17002)
[noreply] [BEAM-13885] Add unit tests to window package (#16971)
[noreply] Merge pull request #16891 from [BEAM-13872] [Playground] Increase test
[noreply] Merge pull request #16912 from [BEAM-13878] [Playground] Increase test
[noreply] Merge pull request #16946 from [BEAM-13873] [Playground] Increase test
[noreply] [BEAM-13951] Update mass_comment.py list of Run commands (#16889)
[noreply] [BEAM-10652] Allow Clustering without Partition in BigQuery (#16578)
[noreply] [BEAM-13857] Add K:V flags for expansion service jars and addresses to
------------------------------------------
[...truncated 325.63 KB...]
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.12
Using cached botocore-1.24.12-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.1-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2678703 sha256=687af014eb77ea2bf8de54dccda5f2f6f4823f74fd68ec97a6ac999da378d33e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.23.0 azure-storage-blob-12.9.0 boto3-1.21.12 botocore-1.24.12 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.1 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.1 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff63a4d0170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff63a4d0950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0304100414.1646406777.841967/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0304100414.1646406777.841967/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0304100414.1646406777.841967/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0304100414.1646406777.841967/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220304151257842945-8655'
createTime: '2022-03-04T15:12:58.911672Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-04_07_12_58-10490538612205770080'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0304100414'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-04T15:12:58.911672Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-04_07_12_58-10490538612205770080]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-04_07_12_58-10490538612205770080
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-04_07_12_58-10490538612205770080?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-04_07_12_58-10490538612205770080 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:02.284Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.243Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.299Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.339Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.373Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.417Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.439Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.467Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.503Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.536Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.564Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.599Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.635Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.676Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.712Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.753Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.910Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:08.979Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:09.018Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:09.058Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-04_07_12_58-10490538612205770080 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:24.387Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:52.635Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:13:52.668Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:14:02.978Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:14:18.402Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:14:18.437Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:49.832Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:49.923Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:49.961Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:50.029Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:50.082Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:50.145Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T15:20:50.200Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T20:00:33.272Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-04_07_12_58-10490538612205770080.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T20:00:33.296Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T20:00:33.463Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-04_07_12_58-10490538612205770080 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T20:00:33.873Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-04T20:00:33.937Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 49m 27s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/5ycxrtqwe6f4c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #511
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/511/display/redirect?page=changes>
Changes:
[Alexey Romanenko] Bump org.mongodb:mongo-java-driver to 3.12.10
[noreply] [BEAM-13973] Link Dataproc Flink master URLs to the InteractiveRunner
[noreply] [BEAM-13925] Turn pr bot on for go prs (#16984)
[Pablo Estrada] Skipping flaky sad-path tests for Spanner changestreams
[noreply] [BEAM-13964] Bump kotlin to 1.6.x (#16882)
[noreply] Merge pull request #16906: [BEAM-13974] Handle idle Storage Api streams
[noreply] Merge pull request #16562 from [BEAM-13051][D] Enable pylint warnings
[noreply] [BEAM-13925] A couple small pr-bot bug fixes (#16996)
[noreply] [BEAM-14029] Add getter, setter for target maven repo (#16995)
[noreply] [BEAM-13903] Improve coverage of metricsx package (#16994)
[noreply] [BEAM-13892] Improve coverage of avroio package (#16990)
------------------------------------------
[...truncated 324.60 KB...]
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.11
Using cached botocore-1.24.11-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2678703 sha256=bc29ebcd7141dab4dbd6cf45ea1461a1dfd1789dee06158ab9092ea68ea630a8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.11 botocore-1.24.11 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.2.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.1 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f09f7cb4ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f09f7cb0710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0303100408.1646320205.046304/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0303100408.1646320205.046304/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0303100408.1646320205.046304/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0303100408.1646320205.046304/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220303151005047253-8578'
createTime: '2022-03-03T15:10:06.604869Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-03_07_10_05-13616824405822405343'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0303100408'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-03T15:10:06.604869Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-03_07_10_05-13616824405822405343]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-03_07_10_05-13616824405822405343
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-03_07_10_05-13616824405822405343?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-03_07_10_05-13616824405822405343 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:09.527Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.285Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.483Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.567Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.650Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.797Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.882Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.926Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:15.975Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.038Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.067Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.123Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.160Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.190Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.224Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.250Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.413Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.494Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.558Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:16.601Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-03_07_10_05-13616824405822405343 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:10:43.506Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:11:02.462Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:11:25.858Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:11:25.886Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.591Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.683Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.708Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.771Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.817Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.871Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T15:17:40.920Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T20:00:34.798Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-03_07_10_05-13616824405822405343.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T20:00:34.832Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T20:00:34.977Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T20:00:35.065Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-03T20:00:35.090Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-03_07_10_05-13616824405822405343 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 52m 30s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/scvff2tdizwf4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #510
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/510/display/redirect?page=changes>
Changes:
[egalpin] Use default context output rather than outputWithTimestamp for
[stranniknm] Palo Alto case study - fix link
[rogelio.hernandez] [BEAM-12777] Removed current docs version redirect
[noreply] Merge pull request #16850: [BEAM-11205] Upgrade Libraries BOM
[noreply] Merge pull request #16484 from [BEAM-13633] [Playground] Implement
[noreply] Add 2022 events blog post (#16975)
[noreply] Clean up Go formatter suggestions (#16973)
[noreply] [BEAM-14012] Add go fmt to Github Actions (#16978)
[noreply] [BEAM-13911] Add basic tests to Go direct runner. (#16979)
[noreply] [BEAM-13960] Add support for more types when converting from between row
------------------------------------------
[...truncated 324.95 KB...]
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.10
Using cached botocore-1.24.10-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2676066 sha256=7313c4eb11edb08cf7c1d37255200ac381aafa73656e01dd2de5f11c5d272818
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.10 botocore-1.24.10 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9e0c562ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9e0c563710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0302100403.1646233128.940844/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0302100403.1646233128.940844/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0302100403.1646233128.940844/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0302100403.1646233128.940844/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220302145848941780-9146'
createTime: '2022-03-02T14:58:49.902830Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-02_06_58_49-6658162998261979915'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0302100403'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-02T14:58:49.902830Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-02_06_58_49-6658162998261979915]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-02_06_58_49-6658162998261979915
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-02_06_58_49-6658162998261979915?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-02_06_58_49-6658162998261979915 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:53.225Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.742Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.779Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.810Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.841Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.884Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.911Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.949Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:58:59.968Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.001Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.037Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.070Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.108Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.142Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.182Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.209Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.328Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.398Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.432Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:00.474Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-02_06_58_49-6658162998261979915 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:07.442Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T14:59:52.147Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:00:14.696Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:00:14.727Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.069Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.150Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.185Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.269Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.328Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.399Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T15:06:53.461Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T20:00:36.788Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-02_06_58_49-6658162998261979915.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T20:00:36.813Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T20:00:36.879Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-02_06_58_49-6658162998261979915 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T20:00:44.889Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-02T20:00:44.930Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 3m 42s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/n735jd6h4b2wo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #509
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/509/display/redirect?page=changes>
Changes:
[noreply] Build wheels for Python 3.9
[noreply] Merge pull request #16892 from [BEAM-13755] [Playground] Scroll the
[noreply] Merge pull request #16880 from [BEAM-13963][Playground] Get bucket name
[noreply] Merge pull request #16870 from [BEAM-13874][Playground] Tag multifile
[noreply] Merge pull request #16910 from [BEAM-13724] [Playground] Get the default
[noreply] [BEAM-14008] Fix incorrect guava import (#16966)
[noreply] Fix ignored exception in BatchSpannerRead. (#16960)
[noreply] [BEAM-13917] Improve coverage of databaseio package (#16956)
[noreply] [BEAM-13925] Add entry files to process new prs and pr updates for PR
[noreply] [BEAM-13899] Improve coverage of debug package (#16951)
[noreply] [BEAM-13907] Improve coverage of textio package (#16937)
[noreply] [BEAM-9150] Fix beam_PostRelease_Python_Candidate (python RC validation
------------------------------------------
[...truncated 325.51 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.9
Using cached botocore-1.24.9-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2676066 sha256=9911c5119aa2c2f61762f295cf0b7ed98d0420505005ac1cfb86f07063c49cc8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.9 botocore-1.24.9 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.1 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9fdb65cef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9fdb658710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0301100414.1646146817.038816/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0301100414.1646146817.038816/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0301100414.1646146817.038816/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0301100414.1646146817.038816/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220301150017039792-3384'
createTime: '2022-03-01T15:00:18.019851Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-03-01_07_00_17-5469778142437960278'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0301100414'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-01T15:00:18.019851Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-03-01_07_00_17-5469778142437960278]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-03-01_07_00_17-5469778142437960278
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-01_07_00_17-5469778142437960278?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-01_07_00_17-5469778142437960278 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:20.516Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.752Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.799Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.834Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.862Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.910Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:21.949Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.004Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.037Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.071Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.109Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.184Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.215Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.278Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.313Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.491Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.558Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.611Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:22.650Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-01_07_00_17-5469778142437960278 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:00:54.261Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:01:08.462Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:01:08.497Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:01:18.793Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:01:30.760Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:01:30.789Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.395Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.480Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.514Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.590Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.636Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.703Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T15:07:40.773Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T20:00:56.976Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-03-01_07_00_17-5469778142437960278.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T20:00:57.015Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T20:00:57.164Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-03-01_07_00_17-5469778142437960278 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T20:01:02.862Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-03-01T20:01:02.912Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 2m 38s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/zjtgpbxdkin3q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #508
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/508/display/redirect>
Changes:
------------------------------------------
[...truncated 324.37 KB...]
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.8
Using cached botocore-1.24.8-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675962 sha256=239c9d9e57a2b5fdd2d98132e7e4d3f18c6d19bdd9e991705284fc3577c505bb
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.8 botocore-1.24.8 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fab9c9c6ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fab9c9c1710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0228100414.1646061266.428341/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0228100414.1646061266.428341/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0228100414.1646061266.428341/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0228100414.1646061266.428341/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220228151426429266-1795'
createTime: '2022-02-28T15:14:27.364483Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-28_07_14_26-10313360134411172018'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0228100414'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-28T15:14:27.364483Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-28_07_14_26-10313360134411172018]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-28_07_14_26-10313360134411172018
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-28_07_14_26-10313360134411172018?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-28_07_14_26-10313360134411172018 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:30.293Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.215Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.288Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.330Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.373Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.431Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.470Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.513Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.542Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.567Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.601Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.631Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.666Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.701Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.745Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:32.781Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:33.025Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:33.091Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:33.131Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:14:33.160Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-28_07_14_26-10313360134411172018 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:15:02.689Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:15:17.698Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:15:38.853Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:15:38.891Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.493Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.588Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.633Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.709Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.783Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.843Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T15:22:02.908Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T20:02:30.897Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-28_07_14_26-10313360134411172018.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T20:02:30.923Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-28_07_14_26-10313360134411172018 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T20:02:31.093Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T20:02:31.469Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-28T20:02:31.501Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 49m 57s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/la7pi763x43ii
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #507
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/507/display/redirect>
Changes:
------------------------------------------
[...truncated 326.23 KB...]
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.8
Using cached botocore-1.24.8-py3-none-any.whl (8.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675962 sha256=f2295260e0392973da3ab3268abaf71bfe623f0d181af8a0e4fd8cb392c967a7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.8 botocore-1.24.8 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa344fd6ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa344fd2710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0227100410.1645974548.013535/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0227100410.1645974548.013535/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0227100410.1645974548.013535/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0227100410.1645974548.013535/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220227150908014526-9615'
createTime: '2022-02-27T15:09:09.014265Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-27_07_09_08-4689869785138723472'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0227100410'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-27T15:09:09.014265Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-27_07_09_08-4689869785138723472]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-27_07_09_08-4689869785138723472
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-27_07_09_08-4689869785138723472?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-27_07_09_08-4689869785138723472 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:12.308Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.334Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.383Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.411Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.450Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.503Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.533Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.565Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.607Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.654Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.688Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.722Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.766Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.801Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-27_07_09_08-4689869785138723472 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:13.887Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:14.132Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:14.305Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:14.383Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:14.435Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:14.482Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:28.084Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:55.765Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:09:55.803Z: JOB_MESSAGE_DETAILED: Resized **** pool to 5, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:10:06.074Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:10:27.647Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:10:27.674Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.133Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.218Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.250Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.326Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.389Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.464Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T15:16:50.555Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T20:00:34.500Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-27_07_09_08-4689869785138723472.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T20:00:34.552Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T20:00:34.691Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T20:00:34.758Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-27T20:00:34.780Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-27_07_09_08-4689869785138723472 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 53m 20s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/aba45qwtrlkbg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #506
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/506/display/redirect?page=changes>
Changes:
[Ankur Goenka] [BEAM-13952] Sickbaying
[noreply] Memoize some objects for timer processing to reduce overhead. (#16207)
[noreply] [BEAM-13965] Use TypeDeserializer if type information is available to
[noreply] [BEAM-13912] Add more coverage for dataflow.go (#16903)
[noreply] [BEAM-12563] swaplevel general function for dataframe and series
[noreply] [BEAM-14001] Update coder.go unit tests (#16952)
[noreply] [BEAM-13910] Improve coverage of gcsx package (#16942)
[noreply] [BEAM-13015] Use a DirectExecutor for state since we are just completing
------------------------------------------
[...truncated 324.36 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.8
Using cached botocore-1.24.8-py3-none-any.whl (8.6 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.1-py3-none-any.whl (54 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675962 sha256=638de91e70962489abf8c7858f05f12dd2148d40266e7974e511d2cf11c2ff03
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.8 botocore-1.24.8 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.3.1 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3be1f29ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3be1f25710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0226100350.1645887264.725242/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0226100350.1645887264.725242/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0226100350.1645887264.725242/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0226100350.1645887264.725242/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220226145424726203-2958'
createTime: '2022-02-26T14:54:25.683505Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-26_06_54_25-18116625603666279724'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0226100350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-26T14:54:25.683505Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-26_06_54_25-18116625603666279724]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-26_06_54_25-18116625603666279724
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-26_06_54_25-18116625603666279724?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-26_06_54_25-18116625603666279724 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:28.298Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.398Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.437Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.474Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.500Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.531Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.625Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.679Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.716Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.746Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.772Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.799Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.835Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.866Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.899Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:30.928Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:31.070Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:31.127Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:31.166Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:31.202Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-26_06_54_25-18116625603666279724 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:54:45.549Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:55:15.960Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:55:38.545Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T14:55:38.619Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:02.789Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:02.862Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:02.888Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:02.944Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:02.992Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:03.071Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T15:02:03.137Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T20:00:32.845Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-26_06_54_25-18116625603666279724.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T20:00:32.868Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T20:00:33.056Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T20:00:33.247Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-26T20:00:33.272Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-26_06_54_25-18116625603666279724 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 8m 2s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/psaxhqx6sp5e6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #505
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/505/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] Revert PR#16253 due errors with plugin flaky-test-handler
[noreply] Fix BoundedQueueExecutor and StreamingDataflowWorker to actually limit
[noreply] [BEAM-1857] Add Neo4jIO (#15916)
[noreply] [BEAM-13767] Migrate serveral portable runner tasks to use configuration
[noreply] [BEAM-13996] Removing 'No cluster_manager is associated with the
[noreply] [BEAM-13906] Improve coverage of errors package (#16934)
[noreply] [BEAM-13886] unit tests for trigger package (#16935)
[noreply] [BEAM-4767] Remove beam- prefix from release script tags (#16899)
[noreply] [BEAM-13866] Add small unit tests to errorx, make boolean assignment
[noreply] [BEAM-13925] Add most of the supporting files for the pr management
[noreply] Merge pull request #16846 from [BEAM-12164]: Add sad path tests for
------------------------------------------
[...truncated 325.27 KB...]
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.7
Using cached botocore-1.24.7-py3-none-any.whl (8.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675832 sha256=7027be248c6c4626e23e4125983d5e384b01f04be36bc7cab3ef6dfb466d17d3
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.7 botocore-1.24.7 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.2 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5e53413ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5e53414710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0225100359.1645802174.356583/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0225100359.1645802174.356583/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0225100359.1645802174.356583/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0225100359.1645802174.356583/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220225151614357506-8769'
createTime: '2022-02-25T15:16:15.503802Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-25_07_16_14-10299782893381272621'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0225100359'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-25T15:16:15.503802Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-25_07_16_14-10299782893381272621]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-25_07_16_14-10299782893381272621
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-25_07_16_14-10299782893381272621?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-25_07_16_14-10299782893381272621 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:19.507Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.562Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.646Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.715Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.825Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.895Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:21.938Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.016Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.078Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.161Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.222Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.283Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.356Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.438Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.533Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.607Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.907Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:22.994Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:23.045Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:23.104Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-25_07_16_14-10299782893381272621 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:16:42.012Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:17:09.540Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:17:31.672Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:17:31.856Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.194Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.262Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.301Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.388Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.449Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.529Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T15:23:49.601Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T18:45:57.517Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T18:45:58.929Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T18:46:21.431Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T20:00:49.328Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-25_07_16_14-10299782893381272621.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T20:00:49.393Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-25_07_16_14-10299782893381272621 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T20:00:49.548Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T20:00:49.691Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-25T20:00:49.732Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 46m 28s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/wxzpijhgdgaic
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #504
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/504/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12645] Fix code-cov flakes due to monorepo. (#16925)
[noreply] [BEAM-13969] Deprecate stringx package (#16884)
[noreply] Add Go badge to ReadMe (#16897)
[noreply] [BEAM-13980] Re-add method gone missing in af2f8ee6 (#16918)
[noreply] [BEAM-13884] Improve mtime package (#16924)
[noreply] Minor: Update Go API doc links (#16932)
[noreply] [BEAM-13218] Re-enable
[noreply] Merge pull request #16857 from [BEAM-13662] [Playground] Support
[noreply] Merge pull request #16826 from [BEAM-13870] [Playground] Increase test
------------------------------------------
[...truncated 325.24 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.6
Using cached botocore-1.24.6-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.0-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.55.0-py2.py3-none-any.whl (212 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675920 sha256=98d9fe859e98d0662ce810807f329c6df588c195b6a7dc484a46bccd67c6f63f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.6 botocore-1.24.6 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.3.0 googleapis-common-protos-1.55.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff148d6c9e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff148d6e200> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0224100407.1645715734.140110/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0224100407.1645715734.140110/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0224100407.1645715734.140110/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0224100407.1645715734.140110/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220224151534141043-7279'
createTime: '2022-02-24T15:15:35.143059Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-24_07_15_34-11697651562631212106'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0224100407'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-24T15:15:35.143059Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-24_07_15_34-11697651562631212106]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-24_07_15_34-11697651562631212106
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-24_07_15_34-11697651562631212106?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-24_07_15_34-11697651562631212106 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:42.112Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.179Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.230Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.257Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.293Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.334Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.369Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.399Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.462Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.500Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.530Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.567Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.595Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.638Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.686Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.849Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.909Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.958Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:15:43.977Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-24_07_15_34-11697651562631212106 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:16:10.915Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:16:27.276Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:16:51.798Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:16:51.818Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.030Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.120Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.154Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.227Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.284Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.364Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T15:23:10.428Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T20:01:05.303Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-24_07_15_34-11697651562631212106.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T20:01:05.331Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T20:01:05.457Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-24_07_15_34-11697651562631212106 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T20:01:16.423Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-24T20:01:16.438Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 47m 27s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/tcdo4pyatrpag
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #503
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/503/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-13796] projection pushdown in BQ IO
[Kyle Weaver] [BEAM-13796] Move test to ReadTest class and correct javadoc for
[Kyle Weaver] [BEAM-13796] Pushdown is not supported on TypedRead#fromQuery.
[noreply] [BEAM-13738] Reenable ignored SQS test after bumping elasticmq for fixed
[noreply] fix build status link (#16907)
[noreply] Merge pull request #16549 from [BEAM-13681][Playground] Refactoring
[noreply] Merge pull request #16732 from [BEAM-13825] [Playground] updated
[noreply] Merge pull request #16683 from [BEAM-13713][Playground] Java graph
[noreply] case study pages - improvements and fixes (#16896)
[noreply] Palo Alto case study (#16915)
------------------------------------------
[...truncated 322.93 KB...]
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.4-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.25.0,>=1.24.4
Using cached botocore-1.24.5-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675864 sha256=a4980accf57a7c10f81f7d807bc49d31b84e60c85e4a937d5035ecfc446904d6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.4 botocore-1.24.5 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc8e536a170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc8e536a950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0223100417.1645628115.892248/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0223100417.1645628115.892248/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0223100417.1645628115.892248/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0223100417.1645628115.892248/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220223145515893164-4912'
createTime: '2022-02-23T14:55:16.894365Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-23_06_55_16-8559673522251069724'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0223100417'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-23T14:55:16.894365Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-23_06_55_16-8559673522251069724]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-23_06_55_16-8559673522251069724
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-23_06_55_16-8559673522251069724?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-23_06_55_16-8559673522251069724 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:20.419Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.054Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.091Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.126Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.147Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.186Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.224Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.253Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.288Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.326Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.359Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.399Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.441Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.492Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.523Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.557Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.746Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.843Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.888Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:21.920Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-23_06_55_16-8559673522251069724 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:55:51.578Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:56:03.047Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:56:26.804Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T14:56:26.841Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:46.820Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:46.920Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:46.952Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:47.021Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:47.095Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:47.192Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-23T15:02:47.302Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy126.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #501
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/501/display/redirect>
Changes:
------------------------------------------
[...truncated 285.33 KB...]
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.1-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Downloading boto3-1.21.4-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.1/132.1 KB 5.1 MB/s eta 0:00:00
Collecting cachetools<5,>=3.1.0
Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting google-apitools<0.5.32,>=0.5.31
Using cached google_apitools-0.5.31-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-2.6.0-py2.py3-none-any.whl (156 kB)
Collecting google-cloud-datastore<2,>=1.8.0
Using cached google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB)
Collecting google-cloud-pubsub<3,>=2.1.0
Using cached google_cloud_pubsub-2.9.0-py2.py3-none-any.whl (219 kB)
Collecting google-cloud-pubsublite<2,>=1.2.0
Using cached google_cloud_pubsublite-1.4.0-py2.py3-none-any.whl (261 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.34.0-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-bigquery-storage>=2.6.3
Using cached google_cloud_bigquery_storage-2.11.0-py2.py3-none-any.whl (172 kB)
Collecting google-cloud-core<2,>=0.28.1
Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.0-py2.py3-none-any.whl (111 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.4
Downloading botocore-1.24.4-py3-none-any.whl (8.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.5/8.5 MB 59.0 MB/s eta 0:00:00
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675864 sha256=42b9077f68c0df04c2faf72b075378ebeb7f410e8bd71a0869d8340992faecf9
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.4 botocore-1.24.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:990 out of 1000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff5a9051b00> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff5a9054320> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0219185704.1645476983.939027/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0219185704.1645476983.939027/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0219185704.1645476983.939027/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0219185704.1645476983.939027/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220221205623940329-9280'
createTime: '2022-02-21T20:56:24.958196Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-21_12_56_24-1346212007575951456'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-8-0219185704'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-21T20:56:24.958196Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-21_12_56_24-1346212007575951456]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-21_12_56_24-1346212007575951456
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-21_12_56_24-1346212007575951456?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-21_12_56_24-1346212007575951456 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:27.845Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.481Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.509Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.539Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.570Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.602Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.627Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.661Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.696Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.754Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.780Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.815Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.845Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.864Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.901Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:28.925Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:29.044Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:29.100Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:29.144Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:29.165Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-21_12_56_24-1346212007575951456 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:54.859Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:59.191Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:56:59.231Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-21T20:57:09.473Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-5' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy121.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-5 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #500
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/500/display/redirect?page=changes>
Changes:
[rogelio.hernandez] [BEAM-13051] Pylint misplaced-bare-raise warning enabled
[Jeff Tapper] Update Java LTS roadmap info on website for Java 17
[Pablo Estrada] Simplify README for new users
[Kyle Weaver] [BEAM-13106] Support Flink 1.14.
[Kyle Weaver] [BEAM-13106] Reuse executor instead of shutting it down mid-test.
[Kyle Weaver] [BEAM-13106] Prevent infinite wait in Flink savepoint test.
[Ismaël Mejía] [BEAM-13202] Fix typos on tests names for VarianceFnTest
[Kenneth Knowles] Disable AfterSynchronizedProcessingTime test on Dataflow
[Ismaël Mejía] [BEAM-13202] Add Coder to CountIfFn.Accum
[Ismaël Mejía] [BEAM-13202] Reuse Count transform code since CountIf is a specific case
[Kenneth Knowles] Add test category UsesProcessingTimeTimers
[Kenneth Knowles] Label tests that need UsesProcessingTimeTimers
[Kenneth Knowles] Exclude UsesProcessingTimeTimers from SamzaRunner tests
[Kyle Weaver] [BEAM-13106] A couple additional fixes to FlinkSavepointTest.
[mmack] [adhoc] Migrate KinesisIOIT to use ITEnvironment for Localstack based IT
[rogelio.hernandez] [BEAM-13051] Added descriptions to Kinesis and PortableRunner exceptions
[noreply] [BEAM-13955] Fix pylint breakage from #16836 (#16867)
[relax] Fix TableRow conversion for the case of fields named "f"
[noreply] Bump dataflow.fnapi_container_version (#16874)
[mmack] [BEAM-13563] Introducing common AWS ClientBuilderFactory to unify
[laraschmidt] Fix final allowskew error to properly handle a large allowedSkew
[noreply] Case studies page improvements (#16702)
[noreply] [BEAM-13946] Add get_dummies(), a non-deferred column operation on
[noreply] [release-2.36.0] Fix pickler argument for 2.36 blog (#16774)
[thiagotnunes] fix: fix bug when retrieving either string or json
[noreply] [adhoc] Avoid using SerializablePipelineOptions for testing to minimize
[noreply] [BEAM-13812] Integrate DataprocClusterManager into Interactive
[noreply] [BEAM-12572] Fix failing python examples tests in Dataflow runner
[noreply] Remove build status from PR (#16902)
------------------------------------------
[...truncated 327.36 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.25.0,>=1.24.3
Using cached botocore-1.24.3-py3-none-any.whl (8.5 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675864 sha256=9185308db5308f3a49fce8a15f54a49b7162fc127bc9456557a823f56be9dcbc
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.3 botocore-1.24.3 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa8a9381c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa8a9382440> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0219101246.1645282818.302987/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0219101246.1645282818.302987/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0219101246.1645282818.302987/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0219101246.1645282818.302987/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220219150018303906-6636'
createTime: '2022-02-19T15:00:19.283650Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-19_07_00_18-15783369163058610502'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0219101246'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-19T15:00:19.283650Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-19_07_00_18-15783369163058610502]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-19_07_00_18-15783369163058610502
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-19_07_00_18-15783369163058610502?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-19_07_00_18-15783369163058610502 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:22.552Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.811Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.840Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.869Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.917Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.951Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:23.995Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.031Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.065Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.099Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.134Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.167Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.191Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.212Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.250Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.291Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-19_07_00_18-15783369163058610502 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.420Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.506Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.542Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:24.573Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:52.355Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:57.319Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 8 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:00:57.340Z: JOB_MESSAGE_DETAILED: Resized **** pool to 8, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:01:07.525Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:01:27.712Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:01:27.740Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.397Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.496Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.533Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.593Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.644Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.702Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T15:07:10.770Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T20:00:41.100Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-19_07_00_18-15783369163058610502.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T20:00:41.124Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T20:00:41.236Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-19_07_00_18-15783369163058610502 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T20:00:41.318Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-19T20:00:41.337Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 2m 10s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/yeyxdc4qt6sgc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #497
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/497/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests.
[Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo.
[noreply] Update 2.36.0 blog post to mention ARM64 support
[stranniknm] [BEAM-13785] playground - enable scio sdk
[noreply] Minor: Disable checker framework in nightly snapshot (#16829)
[artur.khanin] Updated example link
[noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788)
[noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830)
[Kenneth Knowles] Add test for processing time continuation trigger
[noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests
[noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851)
[noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn
[mattcasters] [BEAM-13854] Document casting trick for Avro value serializer in KafkaIO
[noreply] Merge pull request #16838 from [BEAM-13931] - make sure large rows cause
[noreply] Seznam Case Study (#16825)
[noreply] [Website] Apache Hop Case Study (#16824)
[noreply] [BEAM-13694] Force hadoop-hdfs-client in hadoopVersion tests for hdfs
[noreply] [Website] Ricardo - added case study feedback (#16807)
[noreply] Merge pull request #16735 from [BEAM-13827] - fix medium file size
[noreply] Merge pull request #16753 from [BEAM-13837] [Playground] show graph on
------------------------------------------
[...truncated 325.32 KB...]
Collecting boto3>=1.9
Using cached boto3-1.21.0-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.0
Using cached botocore-1.24.0-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.2)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2660308 sha256=ee0e46b71d67a4cf535fd0c207f1901fcec5cfab69b583e9532ae36d9f6659da
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.0 botocore-1.24.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.1 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd80240fd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd802410560> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0216100936.1645027494.829246/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0216100936.1645027494.829246/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0216100936.1645027494.829246/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0216100936.1645027494.829246/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220216160454830317-2502'
createTime: '2022-02-16T16:04:55.795872Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-16_08_04_55-6295900035314208051'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0216100936'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-16T16:04:55.795872Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-16_08_04_55-6295900035314208051]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-16_08_04_55-6295900035314208051
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-16_08_04_55-6295900035314208051?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-16_08_04_55-6295900035314208051 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:58.881Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.613Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.653Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.681Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.713Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.752Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.791Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.826Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.859Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.889Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.924Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:04:59.982Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.007Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.038Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.069Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.102Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.255Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.346Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.383Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:00.454Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-16_08_04_55-6295900035314208051 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:12.355Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:05:50.363Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:06:12.106Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:06:12.149Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.503Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.595Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.630Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.701Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.747Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.811Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T16:12:38.881Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T20:00:43.994Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-16_08_04_55-6295900035314208051.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T20:00:44.023Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T20:00:44.148Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T20:00:44.203Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-16T20:00:44.220Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-16_08_04_55-6295900035314208051 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 57m 49s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/mcjcshy5lu44e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #496
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/496/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests.
[Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo.
[noreply] Update 2.36.0 blog post to mention ARM64 support
[noreply] Minor: Disable checker framework in nightly snapshot (#16829)
[artur.khanin] Updated example link
[noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788)
[noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830)
[Kenneth Knowles] Add test for processing time continuation trigger
[noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests
[noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851)
[noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn
------------------------------------------
[...truncated 286.42 KB...]
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting boto3>=1.9
Using cached boto3-1.21.0-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.0
Using cached botocore-1.24.0-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2660179 sha256=3bfb0b778d8a4364a6e949ee967fea18bb9eb5dc680ebbe87fd85349d6510aca
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.0 botocore-1.24.0 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.1 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:990 out of 1000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2e86053cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2e860554d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0215101431.1644934456.504794/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0215101431.1644934456.504794/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0215101431.1644934456.504794/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0215101431.1644934456.504794/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220215141416505720-5227'
createTime: '2022-02-15T14:14:17.406808Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-15_06_14_17-14600773230651849423'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-8-0215101431'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-15T14:14:17.406808Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-15_06_14_17-14600773230651849423]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-15_06_14_17-14600773230651849423
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-15_06_14_17-14600773230651849423?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-15_06_14_17-14600773230651849423 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:21.585Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:23.931Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:23.963Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:23.991Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.021Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.074Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.106Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.141Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.177Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.206Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.257Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.284Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.320Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.349Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.429Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.826Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.881Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.923Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:24.956Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-15_06_14_17-14600773230651849423 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:14:31.941Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:15:06.235Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:15:26.203Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:15:26.236Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.214Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.304Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.350Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.429Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.485Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.559Z: JOB_MESSAGE_DEBUG: Value "ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-15T14:19:51.630Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy139.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
FATAL: Channel "hudson.remoting.Channel@65326726:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@65326726:apache-beam-jenkins-15": Remote call on apache-beam-jenkins-15 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:994)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1147)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:525)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #495
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/495/display/redirect?page=changes>
Changes:
[akustov] fix name project id from secreton scio deploy action
[alexander.zhuravlev] [BEAM-13775] Fixed bug with run button
[ihr] [BEAM-13836] Fix the answers placeholders locations in the Python katas
[noreply] Merge pull request #16703 from [BEAM-13804][Playground][Bugfix] Add
[noreply] Merge pull request #16611 from [BEAM-13712][Playground] Add graph for
[noreply] Merge pull request #16757 from [BEAM-13655] [Playground] Persist the
------------------------------------------
[...truncated 285.29 KB...]
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting pandas<2.0.0
Using cached pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.54
Using cached botocore-1.23.54-py3-none-any.whl (8.5 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.0)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2659546 sha256=f35e7620420929519e3ac3fc14a7f7685e021e89a6a06f0347551cfb603dfe6c
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.54 botocore-1.23.54 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:990 out of 1000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f72828e2b00> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f72828e3320> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0214100402.1644847778.299508/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0214100402.1644847778.299508/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0214100402.1644847778.299508/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0214100402.1644847778.299508/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220214140938300477-1825'
createTime: '2022-02-14T14:09:39.300043Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-14_06_09_38-5252150491606022935'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-8-0214100402'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-14T14:09:39.300043Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-14_06_09_38-5252150491606022935]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-14_06_09_38-5252150491606022935
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-14_06_09_38-5252150491606022935?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-14_06_09_38-5252150491606022935 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:42.616Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.511Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.543Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.575Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.610Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.642Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.674Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.700Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.720Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.745Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.773Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.808Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.836Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.856Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.889Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:43.921Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:44.054Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:44.111Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:44.146Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:09:44.166Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-14_06_09_38-5252150491606022935 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:10:15.466Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:10:30.261Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:10:54.695Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:10:54.725Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.592Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.654Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.685Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.739Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.791Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.849Z: JOB_MESSAGE_DEBUG: Value "ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T14:16:10.931Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy135.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #494
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/494/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-9195] Bump org.testcontainers to 1.16.3
------------------------------------------
[...truncated 324.42 KB...]
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.1-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.54
Using cached botocore-1.23.54-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.8.2)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2659546 sha256=02d7e70e2b3fbd5deda08d8cb4ed69d2529291f3633a4d57548f72bd6be7c47d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.54 botocore-1.23.54 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.0 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd094d2b200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd094d2b9e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0213100405.1644767644.913257/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0213100405.1644767644.913257/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0213100405.1644767644.913257/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0213100405.1644767644.913257/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220213155404914175-3125'
createTime: '2022-02-13T15:54:05.807107Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-13_07_54_05-12611448153670751544'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0213100405'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-13T15:54:05.807107Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-13_07_54_05-12611448153670751544]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-13_07_54_05-12611448153670751544
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-13_07_54_05-12611448153670751544?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-13_07_54_05-12611448153670751544 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:09.125Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:09.918Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:09.965Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:09.995Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.028Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.062Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.096Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.121Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.168Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.197Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.222Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.246Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.278Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.309Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.343Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.471Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.520Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.556Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:10.589Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-13_07_54_05-12611448153670751544 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:22.552Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:54:55.196Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:55:18.675Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T15:55:18.700Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.189Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.280Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.309Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.352Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.404Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.543Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T16:01:21.639Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T20:00:34.369Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-13_07_54_05-12611448153670751544.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T20:00:34.392Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-13_07_54_05-12611448153670751544 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T20:00:34.576Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T20:00:34.620Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-13T20:00:34.645Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 8m 14s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/nsjtr5jabpyby
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #493
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/493/display/redirect?page=changes>
Changes:
[Carl Yeksigian] Cache bucket matcher regex in GcsPath
[benjamin.gonzalez] [BEAM-12672] Retry flaky tests
[benjamin.gonzalez] [BEAM-12672] Fix spotlessApply
[david.prieto.rivera] Missing contribution
[noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK
[noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776)
[noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for
[laraschmidt] Fixing the log line to properly handle a large allowed skew.
[noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation
[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.
[Valentyn Tymofieiev] Add a container for Python 3.9.
[Valentyn Tymofieiev] Allow job submission with Python 3.9 on Dataflow runner
[Valentyn Tymofieiev] Add Python 3.9 test suites. Keep Dataflow V1 suites unchanged for now.
[Valentyn Tymofieiev] Add py3.9 Github actions suites.
[Valentyn Tymofieiev] Py39 Doc updates.
[Valentyn Tymofieiev] [BEAM-9980] Simplify run_validates_container.sh to avoid branching.
[Valentyn Tymofieiev] Update Cython to a new version that has py39 wheels.
[Valentyn Tymofieiev] [BEAM-13845] Fix comparison with potentially incomparable default
[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.
[Valentyn Tymofieiev] Mark Python 3.9 as supported version.
[noreply] [release-2.36.0][website] Fix github release notes script, header for
[noreply] Use shell to run python for setupVirtualenv (#16796)
[n] BEAM-13159 Update embedded-redis dependency
[Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script.
[noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to
[Valentyn Tymofieiev] [BEAM-13868] Remove gsutil dep from hdfs IT test.
[noreply] [BEAM-13776][Playground] (#16731)
[noreply] [BEAM-13867] Drop NaNs returned by nlargest in flight_delays example
[noreply] Announce Python 3.9 in CHANGES.md (#16802)
[Brian Hulette] Moving to 2.38.0-SNAPSHOT on master branch.
[noreply] [BEAM-11095] Better error handling for iter/reiter/multimap (#16794)
[n] address comments
[noreply] Minor: Add 2.38.0 section to CHANGES.md (#16804)
[noreply] [BEAM-12000] Fix typo in portable Python job definition (#16812)
[noreply] [BEAM-12164]: Fixes SpannerChangeStreamIT (#16806)
[noreply] [BEAM-12572] Fix failures in python examples tests (#16781)
[noreply] [BEAM-13921] filter out debeziumIO test for spark runner (#16815)
[noreply] [BEAM-13855] Skip SpannerChangeStreamOrderedWithinKeyIT and
[noreply] [BEAM-13679] playground - move quick start category to the top (#16808)
[noreply] Update license_script.sh (#16789)
[noreply] [BEAM-13908] [Coverage] Better testing coverage for gcpopts (#16816)
[noreply] Merge pull request #16809 from [BEAM-12164] Added integration test for
[noreply] [BEAM-4032]Support staging binary distributions of dependency packages
[noreply] [BEAM-13834] Increase influxDB persistent storage. (#16817)
[noreply] Minor: Fix link to nexmark benchmarks (#16803)
[noreply] Regenerate python container base_image_requirements.txt (#16832)
------------------------------------------
[...truncated 323.93 KB...]
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.24.0,>=1.23.54
Using cached botocore-1.23.54-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.8.2)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2659546 sha256=13ae48967c714b94e0de2ea3c11b507ddbae01a619ffb25ac3b2078aa3975cc6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.54 botocore-1.23.54 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f31ddc92a70> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f31ddc93290> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0212100604.1644682508.285635/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0212100604.1644682508.285635/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0212100604.1644682508.285635/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0212100604.1644682508.285635/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220212161508286565-3856'
createTime: '2022-02-12T16:15:09.297810Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-12_08_15_08-14518632377486327633'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0212100604'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-12T16:15:09.297810Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-12_08_15_08-14518632377486327633]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-12_08_15_08-14518632377486327633
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-12_08_15_08-14518632377486327633?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-12_08_15_08-14518632377486327633 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.243Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.779Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.809Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.832Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.852Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.882Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.900Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.919Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.942Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:12.996Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.028Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.059Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.087Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.116Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.146Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.167Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.338Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.393Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.441Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:13.464Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-12_08_15_08-14518632377486327633 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:15:44.020Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:16:03.797Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:16:28.144Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:16:28.173Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:02.909Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.002Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.034Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.097Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.142Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.205Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T16:23:03.275Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T20:00:26.830Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-12_08_15_08-14518632377486327633.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T20:00:26.857Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T20:00:26.951Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T20:00:27.042Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-12T20:00:27.065Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-12_08_15_08-14518632377486327633 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 47m 9s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/g7ohdpa7mkrpg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #491
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/491/display/redirect?page=changes>
Changes:
[david.prieto.rivera] Missing contribution
[noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK
[noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776)
[noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for
[noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation
[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.
[Valentyn Tymofieiev] Add a container for Python 3.9.
[Valentyn Tymofieiev] Allow job submission with Python 3.9 on Dataflow runner
[Valentyn Tymofieiev] Add Python 3.9 test suites. Keep Dataflow V1 suites unchanged for now.
[Valentyn Tymofieiev] Add py3.9 Github actions suites.
[Valentyn Tymofieiev] Py39 Doc updates.
[Valentyn Tymofieiev] [BEAM-9980] Simplify run_validates_container.sh to avoid branching.
[Valentyn Tymofieiev] Update Cython to a new version that has py39 wheels.
[Valentyn Tymofieiev] [BEAM-13845] Fix comparison with potentially incomparable default
[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.
[Valentyn Tymofieiev] Mark Python 3.9 as supported version.
[noreply] [release-2.36.0][website] Fix github release notes script, header for
[noreply] Use shell to run python for setupVirtualenv (#16796)
[Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script.
[noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to
[Valentyn Tymofieiev] [BEAM-13868] Remove gsutil dep from hdfs IT test.
[noreply] [BEAM-13776][Playground] (#16731)
[noreply] [BEAM-13867] Drop NaNs returned by nlargest in flight_delays example
[noreply] Announce Python 3.9 in CHANGES.md (#16802)
[Brian Hulette] Moving to 2.38.0-SNAPSHOT on master branch.
[noreply] [BEAM-11095] Better error handling for iter/reiter/multimap (#16794)
------------------------------------------
[...truncated 324.83 KB...]
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.52
Using cached botocore-1.23.52-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.8.2)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2656759 sha256=a6d07acd6703e3fdbe15678a41dd888239e80c7863d6ded26c060aa6698c469c
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.52 botocore-1.23.52 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f349b538710> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f349b538ef0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0210100412.1644509879.254656/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0210100412.1644509879.254656/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0210100412.1644509879.254656/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0210100412.1644509879.254656/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220210161759255587-5321'
createTime: '2022-02-10T16:18:00.479429Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-10_08_17_59-14851034512443797707'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0210100412'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-10T16:18:00.479429Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-10_08_17_59-14851034512443797707]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-10_08_17_59-14851034512443797707
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-10_08_17_59-14851034512443797707?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-10_08_17_59-14851034512443797707 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:07.560Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.193Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.233Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.268Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.301Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.338Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.372Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.406Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.439Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.470Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.503Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.538Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.567Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.593Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.626Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.659Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.823Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.894Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.942Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:09.962Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-10_08_17_59-14851034512443797707 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:18.510Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:49.399Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:18:49.459Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:19:10.189Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:19:10.225Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:19:20.158Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:00.943Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.039Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.075Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.133Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.190Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.251Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-10T16:25:01.336Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy133.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #490
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/490/display/redirect?page=changes>
Changes:
[noreply] Update README.md
[marco.robles] Update README with latest PreCommit Jobs
[marco.robles] Update Postcommit jobs with latest jobs
[marco.robles] Update Performace job tests in readme
[marco.robles] update load job tests with latest updates
[marco.robles] update other jobs test with latest updates
[marco.robles] mismatch links fix
[marco.robles] update trigger phrase for some postCommit jobs
[marco.robles] correct trigger phrases in readme
[marco.robles] add pending jobs to readme
[noreply] Update README.md
[mmack] [BEAM-13246] Add support for S3 Bucket Key at the object level (AWS Sdk
[Pablo Estrada] Output successful rows from BQ Streaming Inserts
[schapman] BEAM-13439 Type annotation for ptransform_fn
[noreply] [BEAM-13606] Fail bundles with failed BigTable mutations (#16751)
[mmack] [adhoc] Remove remaining usage of Powermock from aws2.
[marco.robles] fix broken links in jobs & remove the invalid ones
[Kyle Weaver] Update Dataflow Python dev container images.
[Kiley Sok] Add java 17 to changes
[noreply] [BEAM-12914] Add missing 3.9 opcodes to type inference. (#16761)
[noreply] [BEAM-13321] Initial BigQueryIO externalization. (#16489)
[noreply] [BEAM-13193] Enable process bundle response elements embedding in Java
[noreply] [BEAM-13830] added a debeziumio_expansion_addr flag to GoSDK (#16780)
[noreply] Apply spotless. (#16783)
[Daniel Oliveira] [BEAM-13732] Switch x-lang BigQueryIO expansion service to GCP one.
[noreply] [BEAM-13858] Fix broken github action on :sdks:go:examples:wordCount
[Kiley Sok] add jira for runner v2
[noreply] [BEAM-13732] Go SDK BigQuery IO wrapper. Initial implementation.
[noreply] [BEAM-13732] Add example for Go BigQuery IO wrapper. (#16786)
[noreply] Update CHANGES.md with Go SDK milestones. (#16787)
[noreply] [BEAM-13193] Allow BeamFnDataOutboundObserver to flush elements.
------------------------------------------
[...truncated 325.33 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.24.0,>=1.23.51
Using cached botocore-1.23.51-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.8.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2655298 sha256=88fabb676cb196d9b8e69e80cffb963b0b625dd779884c9884dfdcd857fc1310
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.51 botocore-1.23.51 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fea9fd0d5f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fea9fd0ddd0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0209100836.1644421045.303578/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0209100836.1644421045.303578/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0209100836.1644421045.303578/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0209100836.1644421045.303578/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220209153725304493-3679'
createTime: '2022-02-09T15:37:26.601081Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-09_07_37_25-7689808255495229064'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0209100836'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-09T15:37:26.601081Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-09_07_37_25-7689808255495229064]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-09_07_37_25-7689808255495229064
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-09_07_37_25-7689808255495229064?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-09_07_37_25-7689808255495229064 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:31.353Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.277Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.319Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.344Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.381Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.423Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.458Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.489Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.523Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.548Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.569Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.600Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.626Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.666Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.698Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.721Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.877Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:32.951Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:33.007Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:33.041Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-09_07_37_25-7689808255495229064 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:37:58.030Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:38:17.315Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:38:17.362Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:38:37.832Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:38:41.143Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:38:41.176Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.123Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.220Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.272Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.354Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.424Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.508Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T15:45:09.580Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-09_07_37_25-7689808255495229064 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T20:00:32.304Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-09_07_37_25-7689808255495229064.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T20:00:32.336Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T20:00:32.482Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T20:00:32.592Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-09T20:00:32.630Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 24m 51s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/7itwholrmuyw6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #489
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/489/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12976] Log projection pushdown optimizations.
[benjamin.gonzalez] [BEAM-12572] Change jobs to run as cron jobs
[Ismaël Mejía] [BEAM-13839] Upgrade zstd-jni to version 1.5.2-1
[mmack] [BEAM-13840] Fix usage of legacy rawtypes in AWS modules
[alexander.zhuravlev] [BEAM-13820] Changed color of delete icon in pipeline options dropdown,
[noreply] [BEAM-11971] Revert "Fix timer consistency in direct runner" (#16748)
[noreply] [BEAM-13193] Aggregates fn api outbound data/timers of different
[noreply] [BEAM-13767] Migrate a bundle of grade tasks to use configuration
[noreply] Merge pull request #16653 from [BEAM-12164]: Add integration tests for
[noreply] Merge pull request #16728 from [BEAM-13823] Update docs for SnowflakeIO
[noreply] Merge pull request #16660 from [BEAM-13771][Playground] Send multifile
[noreply] Merge pull request #16646 from [BEAM-13643][Playground] Setup running
[noreply] [BEAM-13015] Add state caching benchmark and move benchmarks to their
[noreply] [BEAM-13419] Check for initialization in dataflow runner (#16765)
[noreply] Merge pull request #16701 from [BEAM-13786] [Playground] [Bugfix] Update
[noreply] Merge pull request #16754 from [BEAM-13838][Playground] Add logs in case
[noreply] [BEAM-13293] consistent naming for expansion service address and flag
[noreply] Merge pull request #16700 from [BEAM-13790][Playground] Change logic of
[noreply] [BEAM-13830] update dependency for debeziumio expansion service (#16743)
[noreply] [BEAM-13761] consistent namings for expansion address in Debezium IO
[noreply] [BEAM-13806] Shutting down SchemaIO expansion services from Go VR
[noreply] [release-2.36.0] Update website/changelog for release 2.36.0 (#16627)
[noreply] [BEAM-13848] Update numpy intersphinx link (#16767)
[noreply] [release-23.6.0] Fix JIRA link for 2.36 blog (#16771)
[noreply] [BEAM-13647] Use role for Go worker binary. (#16729)
------------------------------------------
[...truncated 325.05 KB...]
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.24.0,>=1.23.50
Using cached botocore-1.23.50-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.0-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.8.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2654292 sha256=fda3eccbed496984b91c6ed123bd934fd6730d044889bf2d98f5d3b3bcb04c93
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.50 botocore-1.23.50 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb1c6634dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb1c66365f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0208100629.1644337130.377212/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0208100629.1644337130.377212/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0208100629.1644337130.377212/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0208100629.1644337130.377212/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220208161850378414-1057'
createTime: '2022-02-08T16:18:52.088674Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-08_08_18_50-13291067044290108961'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0208100629'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-08T16:18:52.088674Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-08_08_18_50-13291067044290108961]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-08_08_18_50-13291067044290108961
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-08_08_18_50-13291067044290108961?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-08_08_18_50-13291067044290108961 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:56.774Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.684Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.721Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.757Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.804Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.842Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.869Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.905Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:57.974Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.005Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.072Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.150Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.208Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.258Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.433Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.489Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.535Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:18:58.559Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-08_08_18_50-13291067044290108961 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:19:18.469Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:19:38.878Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:20:04.568Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:20:04.593Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.293Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.405Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.434Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.494Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.542Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.611Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T16:26:25.730Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T20:00:36.538Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-08_08_18_50-13291067044290108961.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T20:00:36.612Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T20:00:36.717Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T20:00:36.878Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T20:00:36.904Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-08_08_18_50-13291067044290108961 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 43m 48s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/btetvz3njbqru
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #488
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/488/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16726 from [BEAM-12164]: Parses change streams
[mmack] [BEAM-13147] Avoid nullness issue during init of AwsModule (AWS Sdk v2)
------------------------------------------
[...truncated 325.39 KB...]
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.0-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Using cached boto3-1.20.49-py3-none-any.whl (131 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.49
Using cached botocore-1.23.49-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.0-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.8.1)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2654153 sha256=29fbf5954504302d5fae7ae851058300ec2a7e9046f41df41a3fc36702542e6d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.49 botocore-1.23.49 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb2c252e680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb2c252ee60> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0207144932.1644263792.504439/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0207144932.1644263792.504439/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0207144932.1644263792.504439/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0207144932.1644263792.504439/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220207195632505357-7246'
createTime: '2022-02-07T19:56:33.815719Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-07_11_56_33-2985028655189763292'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0207144932'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-07T19:56:33.815719Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-07_11_56_33-2985028655189763292]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-07_11_56_33-2985028655189763292
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-07_11_56_33-2985028655189763292?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-07_11_56_33-2985028655189763292 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:42.555Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.211Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.240Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.283Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.321Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.356Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.375Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.404Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.431Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.460Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.488Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.510Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.538Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.559Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.581Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.602Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.732Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.784Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.820Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:43.840Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-07_11_56_33-2985028655189763292 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:56:54.693Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:57:28.082Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:57:51.786Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T19:57:51.803Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.132Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.221Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.255Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.328Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.385Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.436Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-07T20:04:35.515Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T00:11:23.532Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-07_11_56_33-2985028655189763292.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T00:11:23.563Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T00:11:23.711Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T00:11:23.848Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-08T00:11:23.871Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-07_11_56_33-2985028655189763292 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 16m 42s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/arlq5qnwi5cg4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #487
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/487/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13663] Remove unused duplicate option for AWS client configuration
[mmack] [BEAM-13203] Deprecate SnsIO.writeAsync for AWS Sdk v2 due to risk of
[noreply] [BEAM-13828] Fix stale bot (#16734)
[noreply] Merge pull request #16364 from [BEAM-13182] Add diagrams to backend
[noreply] [BEAM-13811] Fix save_main_session arg in tests examples (#16709)
[Kiley Sok] Update beam-master version
[noreply] [BEAM-13015] Calculate exception for closing BeamFnDataInboundObserver2
[noreply] Minor doc tweaks for validating vendoring. (#16747)
[noreply] [BEAM-13686] OOM while logging a large pipeline even when logging level
[noreply] [BEAM-13629] Update URL artifact type for Dataflow Go (#16490)
[noreply] [BEAM-13832] Add automated expansion service start-up to JDBCio (#16739)
[noreply] [BEAM-13831] Add automated expansion service infra into Debezium Read()
[noreply] [BEAM-13821] Add automated expansion service start-up to KafkaIO
[noreply] [BEAM-13799] Created a Dataproc cluster manager for Interactive Beam
[noreply] Merge pull request #16727: [BEAM-11971] remove unsafe Concurrent data
------------------------------------------
[...truncated 324.50 KB...]
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.22.0-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.49
Using cached botocore-1.23.49-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.7.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2654153 sha256=02eb4cbe7ef6edc0c948f27f56be8af6a0731f668a3cb8ff1d7595ea11571732
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.49 botocore-1.23.49 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd44bb8b4d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd44bb8bcb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0205100734.1644077276.483184/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0205100734.1644077276.483184/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0205100734.1644077276.483184/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0205100734.1644077276.483184/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220205160756484134-9739'
createTime: '2022-02-05T16:07:58.164392Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-05_08_07_57-17361755790661034494'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0205100734'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-05T16:07:58.164392Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-05_08_07_57-17361755790661034494]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-05_08_07_57-17361755790661034494
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-05_08_07_57-17361755790661034494?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-05_08_07_57-17361755790661034494 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:10.757Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.758Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.791Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.826Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.858Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.897Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.921Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.947Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:11.974Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.001Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.036Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.071Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.106Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.141Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.176Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.208Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.355Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.417Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.465Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:12.509Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-05_08_07_57-17361755790661034494 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:32.157Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:08:56.341Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:09:19.141Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:09:19.178Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.537Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.616Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.653Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.785Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.848Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.906Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T16:15:40.976Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-05_08_07_57-17361755790661034494 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T20:00:39.817Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-05_08_07_57-17361755790661034494.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T20:00:39.845Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T20:00:39.972Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T20:00:40.048Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-05T20:00:40.068Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 54m 36s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/up2duwy6caksa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #486
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/486/display/redirect?page=changes>
Changes:
[Kiley Sok] Allow Java 17 to be used in SDK
[Kiley Sok] add testing support
[Kiley Sok] Add more testing support for java 17
[Kiley Sok] workaround for jamm
[Kiley Sok] Add Jenkins test for Java 17
[Kiley Sok] Fix jvm hex and skip errorprone
[Kiley Sok] Fix display data for anonymous classes
[Kiley Sok] fix jpms tests
[Kiley Sok] skip zetasql
[Kiley Sok] spotless
[Kiley Sok] spotless
[Kiley Sok] Fix trigger
[Kiley Sok] skip checker framework
[Kiley Sok] fix app name
[Kiley Sok] remove duplicate property check
[Heejong Lee] [BEAM-13813] Add support for URL artifact to extractStagingToPath
[avilovpavel6] Remove Python SQL Test example from catalog
[relax] Fix timer consistency in direct runner
[noreply] [BEAM-13757] adds pane observation in DoFn (#16629)
[Jan Lukavský] Change links to Books from Amazon to Publisher
[noreply] [BEAM-13605] Add support for pandas 1.4.0 (#16590)
[noreply] [BEAM-13761] adds Debezium IO wrapper for Go SDK (#16642)
[noreply] [BEAM-13024] Unify PipelineOptions behavior (#16719)
[noreply] Update sdks/go/pkg/beam/artifact/materialize_test.go
[noreply] Merge pull request #16605 from [BEAM-13634][Playground] Create a
[noreply] Merge pull request #16593 from [BEAM-13725][Playground] Add graph to the
[noreply] Merge pull request #16699 from [BEAM-13789][Playground] Change logic of
[alexander.chermenin] Fixed CSS for Case study page
------------------------------------------
[...truncated 324.70 KB...]
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting boto3>=1.9
Using cached boto3-1.20.48-py3-none-any.whl (131 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.48
Using cached botocore-1.23.48-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.7.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2654126 sha256=57e482eae5b06d1b41e001dc05e3ae7a1043e9fc7d4f0ccf9e953c9414337596
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.48 botocore-1.23.48 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe07156b4d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe07156bcb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0204100400.1643988571.278815/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0204100400.1643988571.278815/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0204100400.1643988571.278815/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0204100400.1643988571.278815/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220204152931279773-3784'
createTime: '2022-02-04T15:29:32.497860Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-04_07_29_31-16480595375916821683'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0204100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-04T15:29:32.497860Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-04_07_29_31-16480595375916821683]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-04_07_29_31-16480595375916821683
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-04_07_29_31-16480595375916821683?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-04_07_29_31-16480595375916821683 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.054Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.634Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.682Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.714Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.750Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.795Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.838Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.879Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.914Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:42.946Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.041Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.075Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.107Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.151Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.182Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.221Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.383Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.460Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.512Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:29:43.551Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-04_07_29_31-16480595375916821683 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:30:07.944Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:30:28.476Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:30:51.200Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:30:51.326Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:08.885Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:08.983Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:09.017Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:09.128Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:09.181Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:09.252Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T15:37:09.321Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T20:00:33.241Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-04_07_29_31-16480595375916821683.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T20:00:33.267Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T20:00:33.408Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T20:00:33.521Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T20:00:33.545Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-04_07_29_31-16480595375916821683 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 32m 56s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/5attqcprhoc24
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #485
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/485/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13737][Playground]
[Robert Bradshaw] Make num-stages counter into an internal counter.
[Kenneth Knowles] [BEAM-13768] Fix NullPointerException in BigQueryStorageSourceBase
[Robert Bradshaw] Avoid packaging avro in the java harness jar.
[mmack] [BEAM-13563] Restructure Kinesis Source for Aws 2 internally to prepare
[noreply] [BEAM-4665] Allow joining a running dataflow pipeline without throwing
[noreply] [BEAM-13801] Add standard coder tests for state backed iterable.
[noreply] [BEAM-13430] Fix provided configuration by removing extendsFrom for
[noreply] [BEAM-12830] Print clearer go version fail message (#16693)
[Jan Lukavský] Add reference to Books to Learning Resources in website
[noreply] Use ThreadLocal for DESERIALIZATION_CONTEXT (#16680)
[noreply] Minor: Add apt update after adding deadsnakes repository in dev env
[noreply] [BEAM-13807] Regenerate container images to get TF 2.8.0 (#16707)
[noreply] [BEAM-13399, BEAM-13683] Eagerly materialize artifacts for automated
[noreply] Merge pull request #16617 from [BEAM-13743] [Playground] Add context
[noreply] Merge pull request #16618 from [BEAM-13744] [Playground] Add context
[noreply] Merge pull request #16698 from [BEAM-13802][Playground] [Bugfix] Clean
[noreply] [BEAM-13293][BEAM-13806] Pipe a SchemaIO flag through Go integration
[noreply] [BEAM-13605] Modify groupby.apply implementation in preparation for
[noreply] Merge pull request #16436 from [BEAM-1330] - DatastoreIO Writes should
------------------------------------------
[...truncated 325.55 KB...]
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.48
Downloading botocore-1.23.48-py3-none-any.whl (8.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.5/8.5 MB 79.6 MB/s eta 0:00:00
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.7.1)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2653688 sha256=a92a33d73aa48a1f9cf02a6f05a9d0d24806509d6e6339fa82f8f25e6f8e3eb1
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.0 azure-storage-blob-12.9.0 boto3-1.20.48 botocore-1.23.48 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff679ab14d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff679ab1cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0203153307.1643920635.877549/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0203153307.1643920635.877549/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0203153307.1643920635.877549/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0203153307.1643920635.877549/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220203203715878499-8403'
createTime: '2022-02-03T20:37:17.218806Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-03_12_37_16-2910388779526126477'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0203153307'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-03T20:37:17.218806Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-03_12_37_16-2910388779526126477]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-03_12_37_16-2910388779526126477
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-03_12_37_16-2910388779526126477?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-03_12_37_16-2910388779526126477 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:23.825Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.526Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.612Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.695Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.722Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.809Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.859Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.907Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:24.977Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.038Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.093Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.141Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.202Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.243Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.275Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.306Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.440Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.573Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.644Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:25.695Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-03_12_37_16-2910388779526126477 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:37:58.264Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:38:09.703Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:38:31.652Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:38:31.686Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.454Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.760Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.793Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.847Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.899Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:00.972Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-03T20:45:01.064Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T00:00:44.359Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-03_12_37_16-2910388779526126477.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T00:00:44.457Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T00:00:44.767Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T00:00:44.883Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-04T00:00:44.909Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-03_12_37_16-2910388779526126477 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 25m 26s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/pvkx26lndjvrw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #484
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/484/display/redirect?page=changes>
Changes:
[career] [BEAM-13734] Support cache directories that use GCS buckets
[noreply] Merge pull request #16655 from [BEAM-12164]: Add retry protection to
[noreply] Merge pull request #16586 from [BEAM-13731] FhirIO: Add support for
[noreply] [BEAM-13011] Adds a link to Multi-language Pipelines Tips wiki page
[noreply] [BEAM-12572] Run python examples on multiple runners (#16154)
[noreply] [BEAM-13574] Large Wordcount (#16455)
[noreply] [BEAM-13293] Refactor JDBC IO Go Wrapper (#16686)
[noreply] Edit license script for Java, add manual licenses for xz (#16692)
------------------------------------------
[...truncated 326.54 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.46
Using cached botocore-1.23.46-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.7.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2652332 sha256=11ae4302297d57b3a0f15a82cc813aabc11793d9eb6bd3bd94002aa82c822d38
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.46 botocore-1.23.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6016da5e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6016da3680> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0202100404.1643815418.120012/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0202100404.1643815418.120012/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0202100404.1643815418.120012/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0202100404.1643815418.120012/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220202152338120921-8081'
createTime: '2022-02-02T15:23:39.467197Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-02_07_23_38-5619460621283576807'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0202100404'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-02T15:23:39.467197Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-02_07_23_38-5619460621283576807]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-02_07_23_38-5619460621283576807
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-02_07_23_38-5619460621283576807?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-02_07_23_38-5619460621283576807 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.105Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.781Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.831Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.866Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.895Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.937Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:44.970Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.003Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.054Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.085Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.182Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.229Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.282Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.323Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.358Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.401Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.592Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.668Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.714Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:23:45.750Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-02_07_23_38-5619460621283576807 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:24:05.399Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:24:31.166Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:24:54.895Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:24:54.925Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.184Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.311Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.357Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.476Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.537Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.647Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T15:31:15.720Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T20:01:14.247Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-02_07_23_38-5619460621283576807.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T20:01:14.271Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T20:01:14.424Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T20:01:14.736Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-02T20:01:14.752Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-02_07_23_38-5619460621283576807 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 39m 31s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/uaqasvqhgzdyu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #483
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/483/display/redirect?page=changes>
Changes:
[daria.malkova] Support SCIO SDK via sbt projects
[samuelw] [BEAM-11648] Share thread pool across RetryManager instances.
[Pablo Estrada] Exclude per-key order tests on Twister2 runner
[mrudary] Generalize S3FileSystem to support multiple URI schemes.
[Heejong Lee] Fix Java SDK container image name for load-tests and nexmark
[daria.malkova] Change executable name fo go tests
[avilovpavel6] Fix java test
[noreply] [BEAM-13769] Skip test_main_session_not_staged_when_using_cloudpickle
[noreply] [BEAM-6744] Support implicitly setting project id in Go Dataflow runner
[noreply] Merge pull request #16493 from [BEAM-13632][Playground] Save catalog
[noreply] Exclude jul-to-slf4j from Spark runner in quickstart POM templates
[noreply] [BEAM-11936] Enable a few errorprone checks that were broken by pinned
[noreply] [BEAM-13780] Add CONTRIBUTING.md pointing to main guide (#16666)
[noreply] [BEAM-13777] Accept cache capacity as input parameter instead of default
[noreply] [BEAM-13051][A] Enable pylint warnings
[noreply] [BEAM-13779] Fix pr labeling (#16665)
[noreply] Merge pull request #16581 from [BEAM-12164]: Add
[noreply] Fix labeler trigger (#16674)
[noreply] [BEAM-13781] Exclude grpc-netty-shaded from gax-grpc's dependency
[noreply] [BEAM-13051] Fixed pylint warnings : raising-non-exception (E0710),
[noreply] [BEAM-13740] Correctly install go before running tests (#16673)
[noreply] [BEAM-12830] Update local Docker env Go version. (#16670)
[noreply] [BEAM-13051][B] Enable pylint warnings
[noreply] [BEAM-13430] Revert Spark libraries in Spark runner to provided (#16675)
[noreply] [BEAM-12240] Add Java 17 support (#16568)
[noreply] [BEAM-13760] Add random component to default python dataflow job name
------------------------------------------
[...truncated 325.01 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.46
Using cached botocore-1.23.46-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.6.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2646201 sha256=1139b409d5e24ab48fb079ba7fd8f1ce150356a59f94ea1bc331589bf74e328e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.46 botocore-1.23.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f195cea5440> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f195cea5c20> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0201100351.1643731016.481660/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0201100351.1643731016.481660/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0201100351.1643731016.481660/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0201100351.1643731016.481660/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220201155656482613-3925'
createTime: '2022-02-01T15:56:58.187616Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-01_07_56_56-17785200789680297984'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0201100351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-01T15:56:58.187616Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-01_07_56_56-17785200789680297984]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-01_07_56_56-17785200789680297984
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-01_07_56_56-17785200789680297984?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-01_07_56_56-17785200789680297984 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:03.413Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.563Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.652Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.705Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.754Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.794Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.834Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.861Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.893Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.918Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.950Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:04.983Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.038Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.070Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.102Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.134Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.285Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.343Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.380Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:05.412Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-01_07_56_56-17785200789680297984 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:14.466Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:57:52.441Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:58:14.587Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T15:58:14.613Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.111Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.206Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.230Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.294Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.383Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.447Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T16:04:36.521Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-01_07_56_56-17785200789680297984 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T20:22:59.798Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-02-01_07_56_56-17785200789680297984.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T20:22:59.821Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T20:22:59.972Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T20:23:00.150Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-01T20:23:00.188Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 27m 54s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/k7dfil5fb4pz2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #482
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/482/display/redirect>
Changes:
------------------------------------------
[...truncated 325.02 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.46
Using cached botocore-1.23.46-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.6.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.11-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645955 sha256=7040bad9ecf2bf9fddaa64174eb8fce79771515532545c770b061001f18135c5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.46 botocore-1.23.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.11 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fde0d1d1440> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fde0d1d1c20> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0131100402.1643644512.372632/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0131100402.1643644512.372632/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0131100402.1643644512.372632/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0131100402.1643644512.372632/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220131155512373530-7109'
createTime: '2022-01-31T15:55:13.502551Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-31_07_55_12-13077186561530274104'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0131100402'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-31T15:55:13.502551Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-31_07_55_12-13077186561530274104]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-31_07_55_12-13077186561530274104
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-31_07_55_12-13077186561530274104?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-31_07_55_12-13077186561530274104 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:18.693Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.425Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.474Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.512Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.542Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.612Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.637Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.677Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.707Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.738Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.781Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.813Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.853Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.883Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.917Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:24.944Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:25.075Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:25.134Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:25.180Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:25.210Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-31_07_55_12-13077186561530274104 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:55:30.050Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:56:12.382Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:56:34.176Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T15:56:34.210Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.530Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.603Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.635Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.705Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.765Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.830Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T16:02:55.906Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T20:00:32.167Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-31_07_55_12-13077186561530274104.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T20:00:32.194Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T20:00:32.400Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T20:00:32.611Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-31T20:00:32.677Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-31_07_55_12-13077186561530274104 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 7m 15s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/dwft56ev7h4c2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #481
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/481/display/redirect>
Changes:
------------------------------------------
[...truncated 325.48 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.46
Using cached botocore-1.23.46-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645956 sha256=0b5337d6513241ef61b72b29b902f743e5e6c12f1068e7c5137a86290f0c0eb6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.46 botocore-1.23.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5c87960170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5c87960950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0130100338.1643557053.748891/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0130100338.1643557053.748891/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0130100338.1643557053.748891/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0130100338.1643557053.748891/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220130153733749799-7702'
createTime: '2022-01-30T15:37:34.908501Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-30_07_37_34-2647910844498018389'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0130100338'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-30T15:37:34.908501Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-30_07_37_34-2647910844498018389]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-30_07_37_34-2647910844498018389
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-30_07_37_34-2647910844498018389?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-30_07_37_34-2647910844498018389 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:40.108Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.126Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.177Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.206Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.251Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.292Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.319Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.341Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.374Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.406Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.430Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.465Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.490Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.510Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.533Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.557Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.685Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.749Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.795Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:37:41.825Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-30_07_37_34-2647910844498018389 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:38:19.029Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:38:23.075Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:38:45.784Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:38:45.820Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.104Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.166Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.198Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.252Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.307Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.370Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T15:45:08.463Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T20:00:33.561Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-30_07_37_34-2647910844498018389.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T20:00:33.586Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T20:00:34.323Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T20:00:34.380Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-30T20:00:34.419Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-30_07_37_34-2647910844498018389 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 24m 47s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/yixharueofm3m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #480
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/480/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-13751] Don't block on gcloud when attempting to get default GCP
[Kyle Weaver] [BEAM-13751] Parameterize wait timeout so test doesn't waste 2s.
[Kyle Weaver] [BEAM-13751] Add comment explaining sleep.
[noreply] Update Python SDK beam-master tags (#16630)
[noreply] Merge pull request #16592 from [BEAM-13722][Playground] Add precompiling
[noreply] Merge pull request #16505 from [BEAM-13527] [Playground] Pipeline
[noreply] [BEAM-13293] XLang Jdbc IO for Go SDK (#16111)
[noreply] [BEAM-10206] Add Go Vet to Github Actions (#16612)
------------------------------------------
[...truncated 327.85 KB...]
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.46
Using cached botocore-1.23.46-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645956 sha256=6679be20ae4023c55836ecd4fb29c2b401172421362beebf5a5c02b7967443c8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.46 botocore-1.23.46 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220126" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f580b00b8c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f580b0110e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0129100412.1643469422.226745/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0129100412.1643469422.226745/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0129100412.1643469422.226745/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0129100412.1643469422.226745/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220129151702227658-6683'
createTime: '2022-01-29T15:17:04.948880Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-29_07_17_02-17629189115231920997'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0129100412'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-29T15:17:04.948880Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-29_07_17_02-17629189115231920997]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-29_07_17_02-17629189115231920997
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-29_07_17_02-17629189115231920997?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-29_07_17_02-17629189115231920997 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:11.193Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.304Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.330Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.350Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.391Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.432Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.453Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.479Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.529Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.561Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.595Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.627Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.652Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.681Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.717Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.740Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.854Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.919Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.958Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:12.982Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-29_07_17_02-17629189115231920997 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:47.914Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:17:59.499Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:18:22.517Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:18:22.542Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.379Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.439Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.474Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.546Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.595Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.644Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T15:24:54.690Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T20:00:34.001Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-29_07_17_02-17629189115231920997.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T20:00:34.022Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T20:00:34.131Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T20:00:34.256Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-29T20:00:34.281Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-29_07_17_02-17629189115231920997 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 45m 21s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/z3a4uc3vqqv3u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #479
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/479/display/redirect?page=changes>
Changes:
[marcin.kuthan] Get rid of unnessecary logs for BigQuery streaming writes in
[dhuntsperger] added GitHub example references to Python multilang quickstart
[mmack] [adhoc] Test S3Options and AwsOptions for Sdk v2
[noreply] [BEAM-13537] Fix NPE in kafkatopubsub example (#16625)
[noreply] [BEAM-13740] update java_tests.yml to remove setup-go, which is
[Heejong Lee] Fix google3 import error
[noreply] [BEAM-12976] Implement Java projection pushdown optimizer. (#16513)
[noreply] Merge pull request #16579 from Revert "Revert "Merge pull request #15863
[noreply] Merge pull request #16606 from [BEAM-13247] [Playground] Embedding
------------------------------------------
[...truncated 324.28 KB...]
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.45
Using cached botocore-1.23.45-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645960 sha256=3fbb1bd99d86afd0cd865b65a751ba4fcd9c0426d8a587c8543c58a872ea429b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.45 botocore-1.23.45 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd2862dd170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd2862dd950> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0128100408.1643386486.331862/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0128100408.1643386486.331862/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0128100408.1643386486.331862/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0128100408.1643386486.331862/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220128161446332888-3117'
createTime: '2022-01-28T16:14:47.496370Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-28_08_14_46-18044990599434518691'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0128100408'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-28T16:14:47.496370Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-28_08_14_46-18044990599434518691]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-28_08_14_46-18044990599434518691
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-28_08_14_46-18044990599434518691?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-28_08_14_46-18044990599434518691 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:57.636Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.575Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.628Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.669Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.701Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.731Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.757Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.789Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.812Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.832Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.857Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.890Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.913Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:58.958Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.026Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.084Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.281Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.337Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.385Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:14:59.403Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-28_08_14_46-18044990599434518691 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:15:35.839Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:15:46.561Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:16:09.407Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:16:09.443Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.022Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.099Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.132Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.201Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.263Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.343Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T16:22:28.413Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T20:00:39.220Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-28_08_14_46-18044990599434518691.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T20:00:39.342Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T20:00:39.716Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T20:00:39.854Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-28T20:00:39.878Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-28_08_14_46-18044990599434518691 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 47m 48s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/tt774wb3z2yse
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #478
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/478/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-13093] Enable JavaUsingPython CrossLanguageValidateRunner test for
[mmack] [BEAM-8807] Add integration test for SnsIO.write (Sdk v1 & v2)
[mmack] [BEAM-13746] Fix deserialization of SSECustomerKey for AWS Sdk v2
[noreply] [BEAM-7928] Allow users to specify worker disk type for Dataflow runner
[noreply] Merge pull request #16534 from [BEAM-13671][Playground] Add backend
[noreply] [BEAM-13271] Bump errorprone to 2.10.0 (#16231)
[noreply] [BEAM-13595] Don't load main session when cloudpickle is used. (#16589)
[Heejong Lee] Update readme for XVR tests
------------------------------------------
[...truncated 323.33 KB...]
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting botocore<1.24.0,>=1.23.44
Using cached botocore-1.23.44-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645960 sha256=f2db9979920d668d83c7a3dd5c9bc5c7784157e08c0ba079cc72dfac3673d5ef
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.44 botocore-1.23.44 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb872243d40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb872240560> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0127100358.1643298853.986632/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0127100358.1643298853.986632/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0127100358.1643298853.986632/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0127100358.1643298853.986632/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220127155413987559-7397'
createTime: '2022-01-27T15:54:15.151153Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-27_07_54_14-17263696082332959364'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0127100358'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-27T15:54:15.151153Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-27_07_54_14-17263696082332959364]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-27_07_54_14-17263696082332959364
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-27_07_54_14-17263696082332959364?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-27_07_54_14-17263696082332959364 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:19.284Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:19.943Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:19.978Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:19.996Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.030Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.073Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.098Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.131Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.158Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.190Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.224Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.259Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.295Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.325Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.350Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.371Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.522Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.580Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.626Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:20.659Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-27_07_54_14-17263696082332959364 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:54:43.284Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:55:02.098Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:55:26.022Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T15:55:26.051Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.363Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.419Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.451Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.529Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.583Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.660Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T16:01:40.737Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T20:13:25.433Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-27_07_54_14-17263696082332959364.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T20:13:25.457Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T20:13:25.594Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T20:13:26.090Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-27T20:13:26.110Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-27_07_54_14-17263696082332959364 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/home/jenkins/.pyenv/versions/3.7.10/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/jenkins/.pyenv/versions/3.7.10/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 21m 10s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/jumcrbvjhb3gi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #477
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/477/display/redirect?page=changes>
Changes:
[artur.khanin] Privacy policy update regarding Apache Beam Playground
[Daniel Oliveira] [BEAM-13321] Fix exception with BigQuery StreamWriter TraceID.
[noreply] [BEAM-13736] Make lifting cache exact. (#16603)
[noreply] Merge pull request #16565 from [BEAM-13692][Playground] Implement
[noreply] Merge pull request #16502 from [BEAM-13650][Playground] Add link for
[noreply] [BEAM-13310] remove call to get offset consumer config, which was rep…
------------------------------------------
[...truncated 323.74 KB...]
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.43
Using cached botocore-1.23.43-py3-none-any.whl (8.5 MB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting libcst>=0.2.5
Using cached libcst-0.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645742 sha256=3e867dba280604c4750956b2de6f13fcff116a33e0bde8d2e9c17fca20d458ea
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.43 botocore-1.23.43 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.5.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.9 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f137690d050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f137690d830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0126100413.1643210062.277143/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0126100413.1643210062.277143/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0126100413.1643210062.277143/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0126100413.1643210062.277143/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220126151422278124-5773'
createTime: '2022-01-26T15:14:26.244934Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-26_07_14_22-16294284007710368634'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0126100413'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-26T15:14:26.244934Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-26_07_14_22-16294284007710368634]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-26_07_14_22-16294284007710368634
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-26_07_14_22-16294284007710368634?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-26_07_14_22-16294284007710368634 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:30.557Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.522Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.561Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.604Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.637Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.678Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.723Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.748Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.779Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.806Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.833Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.866Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.900Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.932Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.965Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:31.999Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:32.178Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:32.236Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:32.273Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:32.298Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-26_07_14_22-16294284007710368634 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:14:44.032Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:15:18.050Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:15:39.025Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:15:39.054Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.425Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.503Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.528Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.612Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.658Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.726Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T15:22:08.792Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T20:00:48.460Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-26_07_14_22-16294284007710368634.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T20:00:48.483Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T20:00:48.617Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T20:00:48.682Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-26T20:00:48.724Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-26_07_14_22-16294284007710368634 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 48m 21s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/fkkre5utvpeko
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #476
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/476/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-13716] Clear before creating a new virtual environment in
[mmack] [BEAM-13653] Make SnsIO.write topicArn optional. If provided, validate
[noreply] [BEAM-10897] Update the fastavro lower bound due to an issue on Windows
[noreply] [BEAM-13605] Update pandas_doctests_test denylists in preparation for
[noreply] Merge pull request #16538 from [BEAM-13676][Playground][Bugfix]Build Of
[noreply] Merge pull request #16582 from [BEAM-13711] [Playground] [Bugfix] Add
[noreply] Merge pull request #16515 from [BEAM-13636] [Playground] Checking the
[ningkang0957] [BEAM-13275] Removed the explicit selenium dependency from setup
[noreply] [BEAM-10206] Deprecate unused shallow cloning functions (#16600)
[noreply] Bump Dataflow container versions (#16602)
[noreply] Improved multi-language pipelines section of the programming guide
[mmack] [BEAM-13510] Don't retry on invalid SQS receipt handles.
------------------------------------------
[...truncated 323.61 KB...]
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.21.1-py2.py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting botocore<1.24.0,>=1.23.42
Using cached botocore-1.23.42-py3-none-any.whl (8.5 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting fasteners>=0.14
Downloading fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645742 sha256=241fe84d65ec0e18905a05dc694eac9e05184c1752bf79d8d0929c5e76d1c0a3
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.42 botocore-1.23.42 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.5.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.8 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3dc3ad5dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3dc3ace5f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0125100359.1643124694.280951/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0125100359.1643124694.280951/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0125100359.1643124694.280951/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0125100359.1643124694.280951/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220125153134281937-1225'
createTime: '2022-01-25T15:31:36.433136Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-25_07_31_34-4618440978733105521'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0125100359'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-25T15:31:36.433136Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-25_07_31_34-4618440978733105521]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-25_07_31_34-4618440978733105521
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-25_07_31_34-4618440978733105521?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-25_07_31_34-4618440978733105521 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:43.839Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.649Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.687Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.714Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.738Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.778Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.805Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.836Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.871Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.907Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:44.964Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.016Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.046Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.068Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.101Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.129Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.283Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.350Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.396Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:31:45.426Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-25_07_31_34-4618440978733105521 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:32:21.636Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:32:28.966Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:32:52.778Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:32:52.807Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.489Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.580Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.626Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.707Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.790Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.858Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T15:39:11.954Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-25_07_31_34-4618440978733105521 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T20:00:33.150Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-25_07_31_34-4618440978733105521.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T20:00:33.176Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T20:00:33.763Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T20:00:33.811Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-25T20:00:33.833Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 30m 53s
15 actionable tasks: 3 executed, 12 up-to-date
Publishing build scan...
https://gradle.com/s/564srootg3egk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #475
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/475/display/redirect>
Changes:
------------------------------------------
[...truncated 324.93 KB...]
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:spotlessGroovy UP-TO-DATE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (3.6.6)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.20.41)
Requirement already satisfied: azure-storage-blob>=12.3.2 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (1.4.31)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.41 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.41)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Requirement already satisfied: wcwidth in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: greenlet!=0.4.17 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645720 sha256=8821a7722334696b1ad7731831b604546990be030c8fa2082a3a1add80d27428
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:990 out of 1000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_SideInput_Dataflow_Batch/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9a8393ccb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9a839354d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0124100343.1643033907.629568/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0124100343.1643033907.629568/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0124100343.1643033907.629568/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-0124100343.1643033907.629568/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220124141827630644-2218'
createTime: '2022-01-24T14:18:29.599982Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-24_06_18_28-5570874313644176671'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-8-0124100343'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-24T14:18:29.599982Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-24_06_18_28-5570874313644176671]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-24_06_18_28-5570874313644176671
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-24_06_18_28-5570874313644176671?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-24_06_18_28-5570874313644176671 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:33.837Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.738Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.783Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.815Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.841Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.884Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.909Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.936Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.959Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:34.984Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.014Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.054Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.093Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.114Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.150Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.186Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.366Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.438Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.481Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:18:35.516Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-24_06_18_28-5570874313644176671 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:09.891Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:24.977Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:25.006Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:35.276Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:47.091Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:19:47.125Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.727Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.790Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.815Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.883Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.933Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:15.992Z: JOB_MESSAGE_DEBUG: Value "ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T14:24:16.048Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:55:46.535Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:55:46.657Z: JOB_MESSAGE_DEBUG: Executing success step success2
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:55:46.715Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:55:46.757Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:55:46.777Z: JOB_MESSAGE_BASIC: Stopping **** pool...
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy123.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #474
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/474/display/redirect>
Changes:
------------------------------------------
[...truncated 375.66 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.6)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.41)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.31)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: botocore<1.24.0,>=1.23.41 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.41)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645720 sha256=8fff12603811dd3f9ae6ab0c60ef8e0ff36ac335cb648ba9c61a538fb598dc5e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc7eabd8dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc7eabd25f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0123100345.1642951363.569618/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0123100345.1642951363.569618/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0123100345.1642951363.569618/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0123100345.1642951363.569618/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220123152243570587-6740'
createTime: '2022-01-23T15:22:44.683328Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-23_07_22_44-15364515628254733072'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0123100345'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-23T15:22:44.683328Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-23_07_22_44-15364515628254733072]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-23_07_22_44-15364515628254733072
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-23_07_22_44-15364515628254733072?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-23_07_22_44-15364515628254733072 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.214Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.820Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.851Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.883Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.911Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.942Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:52.970Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.001Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.036Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.072Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.092Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.118Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.152Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.185Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.214Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.237Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.363Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.418Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.458Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:22:53.484Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-23_07_22_44-15364515628254733072 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:23:10.494Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:23:43.520Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:24:06.187Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:24:06.212Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:14.956Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.015Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.047Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.111Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.164Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.230Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T15:30:15.284Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-23_07_22_44-15364515628254733072 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T20:00:35.034Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-23_07_22_44-15364515628254733072.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T20:00:35.059Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T20:00:35.229Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T20:00:35.302Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-23T20:00:35.324Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 38m 58s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/p6nmhcif53jlg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #473
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/473/display/redirect?page=changes>
Changes:
[ningkang0957] [BEAM-13687] Improved Spanner IO request count metrics
[noreply] [BEAM-10206] Add key for fields in wrapper (#16583)
[noreply] Merge pull request #16530 from Adding JSON support in SpannerIO and
[noreply] [BEAM-13685] Enable users to specify cache directory under Interactive
------------------------------------------
[...truncated 375.33 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.6)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.31)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.41)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.41 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.41)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645720 sha256=16ee7dd82203e40e1a0c62e35c438c0e46a8b3c2d56fc023994131d8e4aea112
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9837776dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f98377705f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0122100404.1642867747.862697/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0122100404.1642867747.862697/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0122100404.1642867747.862697/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0122100404.1642867747.862697/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220122160907863711-5437'
createTime: '2022-01-22T16:09:08.988709Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-22_08_09_08-1685337873478405009'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0122100404'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-22T16:09:08.988709Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-22_08_09_08-1685337873478405009]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-22_08_09_08-1685337873478405009
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-22_08_09_08-1685337873478405009?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-22_08_09_08-1685337873478405009 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:18.163Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.586Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.635Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.667Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.693Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.722Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.759Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.791Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.880Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.913Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.939Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.962Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:19.997Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.026Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.051Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.079Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.212Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.284Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.325Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:20.356Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-22_08_09_08-1685337873478405009 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:09:56.950Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:10:04.624Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:10:26.364Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:10:26.405Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:55.879Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:55.955Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:55.984Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:56.098Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:56.139Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:56.193Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T16:16:56.244Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T20:00:39.023Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-22_08_09_08-1685337873478405009.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T20:00:39.048Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T20:00:39.155Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T20:00:39.278Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-22T20:00:39.294Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-22_08_09_08-1685337873478405009 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 52m 41s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/vkmzzltywrv5w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #472
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/472/display/redirect?page=changes>
Changes:
[thiagotnunes] [BEAM-12164]: Add SDF for reading change stream records
[tuyarer] [BEAM-13577] Beam Select's uniquifyNames function loses nullability of
[sergey.kalinin] Update GH Actions to use proper variables names and proper triggers
[dhuntsperger] edited README and comments in Python multi-lang pipes examples
[Pablo Estrada] Revert "Merge pull request #15863 from [BEAM-13184] Autosharding for
[Pablo Estrada] BEAM-13611 reactivating jdbcio xlang test
[Steve Niemitz] [BEAM-13689] Output token elements when BQ batch writes complete.
[noreply] Merge pull request #16371 from [BEAM-13518][Playground] Beam Playground
[noreply] Update Java FnAPI beam master (#16572)
[noreply] [BEAM-13699] Replace fnv with maphash. (#16573)
[noreply] [BEAM-13693] Bump
[noreply] [BEAM-10206] Remove Fatalf calls in non-test goroutines for
[noreply] [BEAM-13430] Re-add provided configuration (#16552)
[noreply] Merge pull request #16540 from [BEAM-13678][Playground]Update Github
[noreply] Merge pull request #16546 from [BEAM-13661] [BEAM-13704] [Playground]
[noreply] Merge pull request #16369 from [BEAM-13558] [Playground] Hide the Graph
------------------------------------------
[...truncated 375.72 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.31)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.40)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.40 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.40)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644449 sha256=7377f66a7205d00beacda1bca1705cfa6448199989be627c07216a8a72f32bef
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7feeaa533cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7feeaa52f4d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0121100347.1642780543.688859/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0121100347.1642780543.688859/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0121100347.1642780543.688859/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0121100347.1642780543.688859/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220121155543689885-7521'
createTime: '2022-01-21T15:55:44.964311Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-21_07_55_44-15299871232297879492'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0121100347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-21T15:55:44.964311Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-21_07_55_44-15299871232297879492]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-21_07_55_44-15299871232297879492
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-21_07_55_44-15299871232297879492?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-21_07_55_44-15299871232297879492 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:49.365Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.352Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.395Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.427Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.456Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.490Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.519Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.548Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.582Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.607Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.635Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.660Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.687Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.734Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.759Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.791Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:50.942Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:51.001Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:51.037Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:55:51.098Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-21_07_55_44-15299871232297879492 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:24.263Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:35.207Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:35.238Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:55.719Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:58.375Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T15:56:58.406Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:02.660Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:02.756Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:02.777Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:02.874Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:02.940Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:03.010Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T16:03:03.076Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T20:01:19.487Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-21_07_55_44-15299871232297879492.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T20:01:19.510Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T20:01:19.637Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T20:01:19.690Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-21T20:01:19.744Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-21_07_55_44-15299871232297879492 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 42s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/ibg37kxpteuim
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #471
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/471/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13015] Provide caching statistics in the status client. (#16495)
[noreply] [BEAM-13611] Skip test_xlang_jdbc_write (#16554)
[noreply] Merge pull request #16370 from [BEAM-13556] playground - color and
[noreply] Merge pull request #16531 from [BEAM-13567] [playground] Handle run code
[noreply] Merge pull request #16533 from [BEAM-13548] [Playground] Add example
[noreply] Merge pull request #16519 from [BEAM-13639] [Playground] Add
[noreply] Merge pull request #16518 from [BEAM-13619] [Playground] Add loading
[noreply] Merge pull request #16243 from
[noreply] [BEAM-13683] Make cross-language SQL example pipeline (#16567)
[noreply] [BEAM-13688] fixed type in BPG 4.5.3 window section (#16560)
[noreply] Remove obsolete commands from Inventory job. (#16564)
[noreply] Disable logging for memoization test. (#16556)
[noreply] Merge pull request #16472: [BEAM-13697] Add SchemaFieldNumber annotation
[noreply] Merge pull request #16373 from [BEAM-13515] [Playground] Hiding lines in
------------------------------------------
[...truncated 377.22 KB...]
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.30)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.39)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.39 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.39)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644478 sha256=0ba3e80a93fc33e8985fc8cc880156840186954268931d99208e3d1f87f853f3
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa8e1d8bcb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa8e1d874d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0120100348.1642694238.121747/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0120100348.1642694238.121747/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0120100348.1642694238.121747/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0120100348.1642694238.121747/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220120155718122781-5602'
createTime: '2022-01-20T15:57:19.329379Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-20_07_57_18-17828651355084691937'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0120100348'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-20T15:57:19.329379Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-20_07_57_18-17828651355084691937]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-20_07_57_18-17828651355084691937
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-20_07_57_18-17828651355084691937?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-20_07_57_18-17828651355084691937 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:26.708Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.024Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.067Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.096Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.134Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.169Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.198Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.243Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.263Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.298Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.324Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.356Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.400Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.436Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.467Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.502Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.722Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.836Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.895Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:57:28.946Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-20_07_57_18-17828651355084691937 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:03.901Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:16.555Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:16.596Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:26.910Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:43.557Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T15:58:43.584Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.521Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.614Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.647Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.763Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.828Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:01.927Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T16:05:02.026Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T20:00:36.642Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-20_07_57_18-17828651355084691937.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T20:00:36.664Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T20:00:36.772Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T20:00:36.900Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-20T20:00:36.945Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-20_07_57_18-17828651355084691937 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 30s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/d4acjt4oheplc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #470
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/470/display/redirect?page=changes>
Changes:
[mmack] [BEAM-8806] Integration test for SqsIO
[Robert Bradshaw] Bump beam container version.
[alexander.zhuravlev] [BEAM-13680] Fixed code_repository (added pipelineUuid to RunCodeResult
[Robert Bradshaw] Also bump FnAPI container.
[noreply] [BEAM-13616][BEAM-13645] Switch to vendored grpc 1.43.2 (#16543)
[noreply] [BEAM-13616][BEAM-13646] Upgrade vendored calcite to 1.28.0:0.2 (#16544)
[noreply] Merge pull request #16486 from [BEAM-13544][Playground] Add logs to
[noreply] [BEAM-13683] Correct SQL transform schema, fix expansion address
[noreply] Update walkthrough.md (#16512)
[noreply] [BEAM-11808][BEAM-9879] Support aggregate functions with two arguments
[noreply] Merge pull request #16506 from [BEAM-13652][Playground] Send examples'
[noreply] Merge pull request #16322 from [BEAM-13407] [Playground] Preload fonts
[noreply] [BEAM-13665] Make SpannerIO projectID optional again (#16547)
[noreply] [BEAM-13015] Add state caching capability to be used as hint for runners
[noreply] Merge pull request #16309: [BEAM-13503] Set a default value to
------------------------------------------
[...truncated 375.06 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.5.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.38)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.38 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.38)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644379 sha256=c4b03bae62a0887cc3ca2f140608e663ed6a19cae8cb72b777c53629a6f32f89
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20220113" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fafc5f29cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fafc5f254d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0119100407.1642607919.327403/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0119100407.1642607919.327403/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0119100407.1642607919.327403/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0119100407.1642607919.327403/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220119155839328409-7781'
createTime: '2022-01-19T15:58:40.568627Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-19_07_58_39-10206904622500150841'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0119100407'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-19T15:58:40.568627Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-19_07_58_39-10206904622500150841]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-19_07_58_39-10206904622500150841
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-19_07_58_39-10206904622500150841?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-19_07_58_39-10206904622500150841 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:44.903Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:46.882Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.002Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.053Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.120Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.162Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.189Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.214Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.243Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.269Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.293Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.330Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.353Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.383Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.419Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.452Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.578Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.652Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.699Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:58:47.733Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-19_07_58_39-10206904622500150841 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:59:16.488Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:59:33.116Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:59:56.087Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T15:59:56.133Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.283Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.352Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.378Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.433Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.471Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.528Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T16:06:13.577Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-19_07_58_39-10206904622500150841 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T20:00:38.032Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-19_07_58_39-10206904622500150841.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T20:00:38.053Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T20:00:38.226Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T20:00:38.559Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-19T20:00:38.583Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 5s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/m35q6xh6ugvmq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #469
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/469/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13631] Add deterministic SQS message coder to fix reading from SQS
[aydar.zaynutdinov] [BEAM-13641][Playground]
[noreply] Remove jcenter repositories from gradle configuration. (#16532)
[noreply] [BEAM-13430] Remove jcenter which will no longer contain any updates.
[noreply] [BEAM-13616] Update com.google.cloud:libraries-bom to 24.2.0 (#16509)
------------------------------------------
[...truncated 375.68 KB...]
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.37)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.37 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.37)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.17.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644276 sha256=ee26300d2c49037b61b214e53598d255d96b27497407c1e098dcf912544cf016
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f24ab3a3cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f24ab39f4d0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0118100358.1642521488.345460/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0118100358.1642521488.345460/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0118100358.1642521488.345460/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0118100358.1642521488.345460/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220118155808346422-1714'
createTime: '2022-01-18T15:58:09.644199Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-18_07_58_08-1982274383569769158'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0118100358'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-18T15:58:09.644199Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-18_07_58_08-1982274383569769158]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-18_07_58_08-1982274383569769158
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-18_07_58_08-1982274383569769158?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-18_07_58_08-1982274383569769158 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:16.880Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:17.892Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:17.928Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:17.976Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.010Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.038Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.067Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.095Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.123Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.149Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.178Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.202Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.231Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.275Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.304Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.326Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.448Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.508Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.544Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:18.574Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-18_07_58_08-1982274383569769158 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:58:35.519Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:59:09.398Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:59:33.508Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T15:59:33.542Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.488Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.609Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.647Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.699Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.760Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.832Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T16:05:49.892Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T20:00:37.856Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-18_07_58_08-1982274383569769158.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T20:00:37.880Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T20:00:37.925Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T20:00:38.232Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-18T20:00:38.253Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-18_07_58_08-1982274383569769158 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 41s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/eq63qksqxh2iw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #468
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/468/display/redirect?page=changes>
Changes:
[mmack] [BEAM-8806] Integration test for SqsIO using Localstack
[noreply] Merge pull request #16507: [BEAM-13137] Fixes ES utest size flakiness
------------------------------------------
[...truncated 375.13 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.37)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.37 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.37)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644276 sha256=fa4910786cb6becd2398def5543a7afa7dae30d5ec640c554a9f398fbd104a3b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fcb73029b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fcb7302a3b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0117100406.1642435099.323469/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0117100406.1642435099.323469/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0117100406.1642435099.323469/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0117100406.1642435099.323469/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220117155819324423-2319'
createTime: '2022-01-17T15:58:22.484512Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-17_07_58_19-3982341369338567493'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0117100406'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-17T15:58:22.484512Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-17_07_58_19-3982341369338567493]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-17_07_58_19-3982341369338567493
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-17_07_58_19-3982341369338567493?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-17_07_58_19-3982341369338567493 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:28.895Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.649Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.680Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.706Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.732Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.760Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.785Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.840Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.854Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.880Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.904Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.923Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.949Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.976Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:34.994Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:35.017Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:35.160Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:35.205Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:35.252Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:35.274Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-17_07_58_19-3982341369338567493 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:58:52.911Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:59:21.747Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:59:43.888Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T15:59:43.913Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.704Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.809Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.834Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.894Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.935Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:15.983Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T16:06:16.067Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-17_07_58_19-3982341369338567493 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T20:00:34.486Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-17_07_58_19-3982341369338567493.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T20:00:34.517Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T20:00:34.594Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T20:00:34.829Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-17T20:00:34.849Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 23s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://scans.gradle.com/s/v6bc42f6loplk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #467
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/467/display/redirect>
Changes:
------------------------------------------
[...truncated 375.47 KB...]
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.37)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.37 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.37)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644276 sha256=04e1d23676ffc14a3045306e6b9e68518a37acc9aecbf5ffc6f9835e69836d4b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8a74dbfb90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8a74dc03b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0116100349.1642346933.566230/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0116100349.1642346933.566230/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0116100349.1642346933.566230/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0116100349.1642346933.566230/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220116152853567328-1069'
createTime: '2022-01-16T15:28:57.591613Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-16_07_28_54-6118567967565275592'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0116100349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-16T15:28:57.591613Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-16_07_28_54-6118567967565275592]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-16_07_28_54-6118567967565275592
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-16_07_28_54-6118567967565275592?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-16_07_28_54-6118567967565275592 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:04.091Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.005Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.043Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.081Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.115Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.157Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.190Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.215Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.249Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.279Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.317Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.351Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.387Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.429Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.459Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.498Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.642Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.709Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.744Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:05.771Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-16_07_28_54-6118567967565275592 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:23.057Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:29:56.869Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:30:20.112Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:30:20.149Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.071Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.123Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.157Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.209Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.265Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.316Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T15:36:30.383Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-16_07_28_54-6118567967565275592 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T20:00:33.543Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-16_07_28_54-6118567967565275592.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T20:00:33.571Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T20:00:33.665Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T20:00:33.895Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-16T20:00:33.916Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 32m 45s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://scans.gradle.com/s/wulypmgabnohc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #466
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/466/display/redirect?page=changes>
Changes:
[daria.malkova] Split builder into several builder for each step of pipeline execution
[Valentyn Tymofieiev] Provide API to check whether a hint is known.
[Valentyn Tymofieiev] [BEAM-12558] Fix doc typo.
[arietis27] [BEAM-13400] JDBC IO does not support UUID and JSONB PostgreSQL types
[jrmccluskey] [BEAM-10206] Resolve go vet errors in protox package
[noreply] Merge pull request #16482 from [BEAM-13429][Playground] Add builder for
[noreply] [BEAM-13590] Fix abc imports from collections (#15850)
[jrmccluskey] Fix staticcheck errors in transforms directory
[jrmccluskey] Remove unnecessary fmt.Sprintf() in partition.go
[jrmccluskey] Replace bytes.Compare() with bytes.Equal() in test cases
[jrmccluskey] Replace string(buf.Bytes()) with buf.String() in coder_test.go
[jrmccluskey] Remove unnecessary blank identifier assignment in harness.go
[jrmccluskey] fix capitalized error strings in expansionx
[jrmccluskey] Clean up string cast of bytes in vet.go and corresponding tests
[jrmccluskey] Remove unnecessary fmt call in universal.go
[Robert Bradshaw] Remove tab from source.
[noreply] Redirecting cross-language transforms content (#16504)
[noreply] doc tweaks (#16498)
[noreply] [BEAM-12621] - Update Jenkins VMs to modern Ubuntu version (#16457)
[noreply] [BEAM-13664] Fix Primitives hashing benchmark (#16523)
------------------------------------------
[...truncated 375.93 KB...]
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.37)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.37 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.37)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644276 sha256=de19c23e66451770d6a86277f6a2708b95df76ce62893ee9aec0a2eab012f5a3
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8c446b4b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8c446b53b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0115100401.1642259470.589889/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0115100401.1642259470.589889/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0115100401.1642259470.589889/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0115100401.1642259470.589889/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220115151110590883-2406'
createTime: '2022-01-15T15:11:11.822163Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-15_07_11_11-15577924823703003815'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0115100401'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-15T15:11:11.822163Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-15_07_11_11-15577924823703003815]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-15_07_11_11-15577924823703003815
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-15_07_11_11-15577924823703003815?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-15_07_11_11-15577924823703003815 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:15.957Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:16.942Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:16.984Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.007Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.035Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.101Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.120Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.145Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.169Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.213Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.263Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.288Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.322Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.342Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.363Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.397Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.571Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.659Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.693Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:17.726Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-15_07_11_11-15577924823703003815 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:11:34.148Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:12:00.852Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:12:24.832Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:12:24.863Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.555Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.623Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.655Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.735Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.779Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.856Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T15:18:38.934Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T20:00:33.755Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-15_07_11_11-15577924823703003815.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T20:00:33.777Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T20:00:33.916Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T20:00:33.990Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-15T20:00:34.011Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-15_07_11_11-15577924823703003815 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 50m 33s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://scans.gradle.com/s/mmoin64f4xiwi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #465
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/465/display/redirect?page=changes>
Changes:
[blais] python sdk examples: Fixed typo in wordcount example.
[noreply] [BEAM-13480] Increase pipeline timeout for
[noreply] Stronger typing inference for CoGBK. (#16465)
[noreply] [BEAM-12464] Change ProtoSchemaTranslator beam schema creation to match
[noreply] Introduce the notion of a JoinIndex for fewer shuffles. (#16101)
[noreply] Merge pull request #16467 from [BEAM-12164]: SpannerIO
[noreply] Merge pull request #16385 from [BEAM-13535] [Playground] add cancel
[noreply] Merge pull request #16485 from [BEAM-13486] [Playground] For unit tests
[heejong] [BEAM-13455] Remove duplicated artifacts when using multiple
[noreply] [BEAM-12572] Run java examples on multiple runners (#16450)
------------------------------------------
[...truncated 378.82 KB...]
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.32.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.11.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.35)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: botocore<1.24.0,>=1.23.35 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.35)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2644172 sha256=2d1141bb60be7a305a07ff51c4fcd045cdd6a276a2940e89fa89688b4e6c5ede
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f17e25280e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f17e25288c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0114100403.1642175523.847207/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0114100403.1642175523.847207/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0114100403.1642175523.847207/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0114100403.1642175523.847207/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220114155203848164-2413'
createTime: '2022-01-14T15:52:04.952978Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-14_07_52_04-10055929987482604323'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0114100403'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-14T15:52:04.952978Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-14_07_52_04-10055929987482604323]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-14_07_52_04-10055929987482604323
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-14_07_52_04-10055929987482604323?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-14_07_52_04-10055929987482604323 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:09.305Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:09.931Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:09.970Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:09.991Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.053Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.094Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.121Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.144Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.175Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.201Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.236Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.258Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.284Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.313Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.345Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.378Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.532Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.600Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.637Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:10.666Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-14_07_52_04-10055929987482604323 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:42.717Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:52:54.057Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:53:17.466Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:53:17.501Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.066Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.151Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.175Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.234Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.283Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.335Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T15:59:17.392Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T20:00:33.693Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-14_07_52_04-10055929987482604323 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T20:00:33.837Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T20:00:33.896Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-14T20:00:33.916Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 9m 36s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (1 retry remaining)...
https://scans.gradle.com/s/iu2jhiqddubgs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #464
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/464/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13638] Datatype of timestamp fields in SqsMessage for AWS IOs for
[noreply] Merge pull request #16469 from [BEAM-13623][Playground] [Bugfix] During
[noreply] Merge pull request #16149 from [BEAM-13113] [Playground] playground
[noreply] Merge pull request #16363 from [BEAM-13557] [Playground] show code
[noreply] Merge pull request #16374 from [BEAM-13398][Playground] Split LifeCycle
[noreply] [BEAM-13616][BEAM-13646] Update vendored calcite 1.28.0 with protobuf
[chamikaramj] Adds several example multi-language Python pipelines
[noreply] Merge pull request #16325 from [BEAM-13471] [Playground] Tag existing
[noreply] [BEAM-13399] Move service liveness polling to Runner type (#16487)
------------------------------------------
[...truncated 375.44 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.34)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.34 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.34)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2642746 sha256=8493970e27696c52ea43c5a57ad15e4b43851c9164db73eec4dc3f0e4d69385c
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb655732ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb65572f710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0113100356.1642088311.476858/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0113100356.1642088311.476858/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0113100356.1642088311.476858/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0113100356.1642088311.476858/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220113153831477831-9300'
createTime: '2022-01-13T15:38:32.655112Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-13_07_38_31-16947169397947401765'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0113100356'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-13T15:38:32.655112Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-13_07_38_31-16947169397947401765]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-13_07_38_31-16947169397947401765
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-13_07_38_31-16947169397947401765?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-13_07_38_31-16947169397947401765 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:37.515Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.623Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.655Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.688Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.720Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.776Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.806Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.842Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.873Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.910Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.943Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:38.976Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.012Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.046Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.077Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.111Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.258Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.329Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.375Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:39.411Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-13_07_38_31-16947169397947401765 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:38:52.172Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:39:30.978Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:39:51.835Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:39:51.875Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.161Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.276Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.306Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.378Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.433Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.504Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T15:46:31.564Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-13_07_38_31-16947169397947401765 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T20:00:25.474Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-13_07_38_31-16947169397947401765.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T20:00:25.499Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T20:00:26.922Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T20:00:26.986Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-13T20:00:27.005Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 23m 1s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/zcbytcvg5qwye
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #463
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/463/display/redirect?page=changes>
Changes:
[ningkang0957] [BEAM-13602] Prevented metrics gathering from failing bigtable io
[ningkang0957] make the code more pythonic
[mmack] [BEAM-13243][BEAM-8374] Add support for missing PublishResponse fields
[Robert Bradshaw] Optional args and kwargs for named external transforms.
[mmack] [adhoc] Run spotlessApply on java examples to fix master
[noreply] [BEAM-13628] Update SideInputCache to use full Transform and
[noreply] [BEAM-13432] Skip ExpansionService creation in Job Server (#16222)
[noreply] [BEAM-13616] Initial files for vendored gRPC 1.43.2 (#16460)
------------------------------------------
[...truncated 375.00 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.33)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.33 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.33)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2642127 sha256=f23ae9aa4c16673080e7782bd371477fbb97ab1eabd0984b839ae187aedd1c55
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff175f1aef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff175f19710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0112100347.1642002753.045002/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0112100347.1642002753.045002/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0112100347.1642002753.045002/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0112100347.1642002753.045002/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220112155233045947-4706'
createTime: '2022-01-12T15:52:34.368533Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-12_07_52_33-11718063493257488180'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0112100347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-12T15:52:34.368533Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-12_07_52_33-11718063493257488180]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-12_07_52_33-11718063493257488180
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-12_07_52_33-11718063493257488180?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-12_07_52_33-11718063493257488180 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:38.045Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:38.985Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.026Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.057Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.085Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.114Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.135Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.163Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.190Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.224Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.254Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.282Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.313Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.366Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.396Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-12_07_52_33-11718063493257488180 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.424Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.581Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.639Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.675Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:39.696Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:52:50.011Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:53:23.229Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:53:48.160Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T15:53:48.195Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.051Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.118Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.149Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.242Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.300Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.381Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T16:00:11.438Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-12_07_52_33-11718063493257488180 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T20:00:26.206Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-12_07_52_33-11718063493257488180.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T20:00:26.231Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T20:00:26.357Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T20:00:26.460Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-12T20:00:26.484Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 9m 1s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/6z4w2ydh6f33g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #462
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/462/display/redirect?page=changes>
Changes:
[relax] don't close streams in finalize
[noreply] Loosen typing extensions bound
[Alexey Romanenko] [BEAM-4868] Bump com.amazonaws to 1.12.135
[noreply] [BEAM-12092] Bump jedis to version 4.0.1 (#16287)
[noreply] [BEAM-13534] Add automated port polling to expansion service runner if
[noreply] Merge pull request #16344 from [BEAM-13536][Playground][Bugfix] CI step
[noreply] Merge pull request #16359 from [BEAM-13545][Playground] Add
[noreply] Merge pull request #16384 from [BEAM-13308] [Playground] Getting
[noreply] Merge pull request #16306 from [BEAM-13447] [Playground] Add filling of
[noreply] Merge pull request #16361 from [BEAM-13543][Playground] Add logic of
[noreply] [BEAM-12562] Dataframe pipe implementation (#16256)
[noreply] Merge pull request #16338 from [BEAM-13528][Playground] Add liveness
[noreply] [BEAM-13626] Remap expanded outputs after merging. (#16471)
[noreply] Merge pull request #16147 from [BEAM-13359] [Playground] Tag existing
[noreply] [BEAM-3221] Improve documentation in model pipeline protos (#16474)
[noreply] [BEAM-13614] Add OnWindowExpiration support to the Java SDK harness and
[noreply] Merge pull request #16156 from [BEAM-13391] Fix temporary file format in
------------------------------------------
[...truncated 375.97 KB...]
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.3)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.0.1)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.32)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.32 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.32)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2641903 sha256=ab9455b6f7c07057036f65fb7ea8645478816f2a87ea984ee59df1275644c425
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd8da327ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd8da328710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0111100346.1641916212.378356/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0111100346.1641916212.378356/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0111100346.1641916212.378356/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0111100346.1641916212.378356/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220111155012379335-2590'
createTime: '2022-01-11T15:50:13.682009Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-11_07_50_12-11048867283756975815'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0111100346'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-11T15:50:13.682009Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-11_07_50_12-11048867283756975815]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-11_07_50_12-11048867283756975815
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-11_07_50_12-11048867283756975815?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-11_07_50_12-11048867283756975815 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:17.894Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.590Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.650Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.696Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.728Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.803Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.836Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.882Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.915Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:18.979Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.003Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.035Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.072Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.100Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.134Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.214Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.422Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.554Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.638Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:19.700Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-11_07_50_12-11048867283756975815 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:50:42.217Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:51:03.319Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:51:28.006Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:51:28.042Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.136Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.306Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.374Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.465Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.519Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.584Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T15:57:55.670Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T20:00:34.147Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-11_07_50_12-11048867283756975815.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T20:00:34.173Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T20:00:34.325Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T20:00:34.448Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-11T20:00:34.468Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-11_07_50_12-11048867283756975815 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 11m 31s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/ry35z76wy3jys
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #461
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/461/display/redirect?page=changes>
Changes:
[Alexey Romanenko] [BEAM-8727] Bump software.amazon.awssdk to 2.17.106
------------------------------------------
[...truncated 376.83 KB...]
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.31)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.31)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2640546 sha256=489176387dd13243378680cf60a4998b0b15e43ae397181dc38e955411f910d7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7efcc5994560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7efcc5994d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0110100342.1641830545.949163/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0110100342.1641830545.949163/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0110100342.1641830545.949163/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0110100342.1641830545.949163/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220110160225950144-7153'
createTime: '2022-01-10T16:02:27.117630Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-10_08_02_26-12562281465829231772'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0110100342'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-10T16:02:27.117630Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-10_08_02_26-12562281465829231772]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-10_08_02_26-12562281465829231772
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-10_08_02_26-12562281465829231772?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-10_08_02_26-12562281465829231772 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:31.689Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:32.896Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:32.945Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:32.971Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.005Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.034Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.062Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.086Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.121Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.150Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.182Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.215Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.248Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.280Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.312Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.343Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.484Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.539Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.579Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:02:33.624Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-10_08_02_26-12562281465829231772 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:03:09.325Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:03:17.902Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:03:40.875Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:03:40.904Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.657Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.770Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.798Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.854Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.899Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:54.962Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T16:09:55.023Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-10_08_02_26-12562281465829231772 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T20:00:29.706Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-10_08_02_26-12562281465829231772.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T20:00:29.729Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T20:00:29.844Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T20:00:29.940Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-10T20:00:29.964Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 59m 15s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/4cgubruyrosds
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #460
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/460/display/redirect>
Changes:
------------------------------------------
[...truncated 375.06 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.31)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.31)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2640546 sha256=6a4caf3af5cd7e0f52ac406726d08d494f6050c459b90332073dd64e05b569c9
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6ace6da560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6ace6dad40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0109100400.1641743111.627047/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0109100400.1641743111.627047/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0109100400.1641743111.627047/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0109100400.1641743111.627047/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220109154511628031-6721'
createTime: '2022-01-09T15:45:12.772545Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-09_07_45_12-12445220851697313317'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0109100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-09T15:45:12.772545Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-09_07_45_12-12445220851697313317]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-09_07_45_12-12445220851697313317
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-09_07_45_12-12445220851697313317?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-09_07_45_12-12445220851697313317 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:17.043Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:17.950Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:17.985Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.018Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.057Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.086Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.120Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.152Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.185Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.219Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.250Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.304Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.330Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.366Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.387Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.415Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.552Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.604Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.651Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:18.673Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-09_07_45_12-12445220851697313317 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:45:41.430Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:46:04.200Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:46:29.644Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:46:29.680Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:11.923Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:11.984Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:12.009Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:12.069Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:12.114Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:12.175Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T15:53:12.247Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T20:00:31.656Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-09_07_45_12-12445220851697313317.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T20:00:31.679Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T20:00:31.810Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T20:00:32.006Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-09T20:00:32.026Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-09_07_45_12-12445220851697313317 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 16m 31s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/3zwfn5tuc57nm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #459
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/459/display/redirect?page=changes>
Changes:
[Alexey Romanenko] [adhoc] Fix BigTableIO description
[noreply] [BEAM-13015] Remove dead code now that all instances have migrated to
[noreply] [BEAM-13386] Add RLock support for cloudpickle (#16250)
[danthev] Fix overflow
------------------------------------------
[...truncated 374.91 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.9)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.31)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.31)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.3.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2640546 sha256=f208b838dd065cb1af82791fa5616f53ca80feae4141bce79a1781e27042607e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f87ba313440> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f87ba313c20> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0108100350.1641657342.827017/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0108100350.1641657342.827017/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0108100350.1641657342.827017/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0108100350.1641657342.827017/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220108155542827955-2501'
createTime: '2022-01-08T15:55:43.914809Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-08_07_55_43-824258529659229483'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0108100350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-08T15:55:43.914809Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-08_07_55_43-824258529659229483]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-08_07_55_43-824258529659229483
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-08_07_55_43-824258529659229483?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-08_07_55_43-824258529659229483 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:47.945Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.611Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.642Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.667Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.702Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.733Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.759Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.792Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.826Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.859Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.894Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.927Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.963Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:48.995Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.029Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.053Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.181Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.240Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.300Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:55:49.332Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-08_07_55_43-824258529659229483 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:56:19.882Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:56:34.417Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:56:59.470Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T15:56:59.495Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:27.901Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.012Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.050Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.110Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.164Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.228Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T16:03:28.289Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T20:00:40.125Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-08_07_55_43-824258529659229483.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T20:00:40.152Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-08_07_55_43-824258529659229483 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T20:00:40.281Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T20:00:40.360Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-08T20:00:40.380Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 6m 5s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/mw3npujwftg6q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #458
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/458/display/redirect?page=changes>
Changes:
[heejong] [BEAM-13091] Generate missing staged names from hash for Dataflow runner
[heejong] add test
[arietis27] [BEAM-13604] NPE while getting null from BigDecimal column
[noreply] Fixed empty labels treated as wildcard when matching cache files
[noreply] [BEAM-13570] Remove erroneous compileClasspath dependency. (#16438)
[noreply] [BEAM-13015] Plumb through process wide and bundle cache through the
[noreply] [BEAM-13015] Cache the state backed iterable used for large GBK results.
[noreply] Fix formatting/alignment (#16443)
[noreply] Merge pull request #16183 from [BEAM-13427] [Playground] show logs for
[noreply] [BEAM-10277] re-write encoding position tests to declare schema protos
[noreply] Update local_env_tests.yml (#16444)
[noreply] [BEAM-13574] Filesystem abstraction Rename support (#16428)
[noreply] [BEAM-13597] Setup Go in github actions (#16446)
[noreply] Merge pull request #16161 from [BEAM-12164] Add Spanner Partition
[noreply] Merge pull request #16203 from [BEAM-12164] Add Spanner Change Stream
------------------------------------------
[...truncated 375.45 KB...]
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.30)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: botocore<1.24.0,>=1.23.30 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.30)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.3.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2640160 sha256=7f80e628514ef197e3f0c9148050fe368fd8bd3e3aa7858c0468375070a427da
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fede7587b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fede75883b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0107100358.1641569686.743885/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0107100358.1641569686.743885/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0107100358.1641569686.743885/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0107100358.1641569686.743885/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220107153446744856-4588'
createTime: '2022-01-07T15:34:47.934954Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-07_07_34_47-9017469709164642106'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0107100358'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-07T15:34:47.934954Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-07_07_34_47-9017469709164642106]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-07_07_34_47-9017469709164642106
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-07_07_34_47-9017469709164642106?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-07_07_34_47-9017469709164642106 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:08.886Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.571Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.615Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.638Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.661Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.699Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.716Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.742Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.766Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.813Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.867Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.922Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:09.990Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.032Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.066Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.089Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.210Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.303Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.337Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:10.357Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-07_07_34_47-9017469709164642106 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:35:38.950Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:36:00.597Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:36:25.934Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:36:25.959Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:56.659Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:56.762Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:56.798Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:56.862Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:56.926Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:57.016Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T15:42:57.072Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T20:00:35.309Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-07_07_34_47-9017469709164642106.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T20:00:35.336Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T20:00:35.467Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T20:00:35.721Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-07T20:00:35.739Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-07_07_34_47-9017469709164642106 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 27m 1s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/bnj3n63hu54nk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #457
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/457/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13565][Playground]
[relax] update bom
[zyichi] Upgrade python library versions in base_image_requirements.txt
[noreply] [BEAM-13567] Consolidate runner flag definition. (#16426)
[noreply] [BEAM-13601] Don't cache Row types for a schema. (#16427)
[noreply] [BEAM-13430] Re-enable checkerframework (#16429)
[noreply] [BEAM-13430] Ensure that testRuntimeMigration depends on "default"
[noreply] Merge pull request #16277 from [BEAM-13124][Playground] Create readiness
[noreply] Merge pull request #16314 from [BEAM-13260][Playground]Implement setup
[noreply] Merge pull request #16383 from [BEAM-13566][Playground] Add logic of
[noreply] Merge pull request #16365 from [BEAM-13559][Playground] Remove tag in
[noreply] Merge pull request #16360 from [BEAM-13546][Playground] Update nginx
[noreply] Merge pull request #16192 from [BEAM-13395] [Playground] Tag katas
[noreply] Merge pull request #16254 from [BEAM-13249][Playground] Security – Mock
[ningkang0957] [BEAM-12879] Prevented missing permission from failing GCS I/O
[noreply] Merge pull request #16347: fix: move connector to use v1 BigQuery
[noreply] [BEAM-13603] Fix bug in apache_beam.utils.Shared (#16437)
[noreply] [BEAM-10345] Add an import guard to support recent google-cloud-spanner
------------------------------------------
[...truncated 375.09 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.1)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.29)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.29 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.29)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2640111 sha256=26ad886076b51be81ff22e50222681edc68276686e2ee919f9e89e09cfdd6e36
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fbbfb1d0b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fbbfb1543b0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0106100357.1641484434.956549/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0106100357.1641484434.956549/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0106100357.1641484434.956549/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0106100357.1641484434.956549/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220106155354957531-9723'
createTime: '2022-01-06T15:53:56.134138Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-06_07_53_55-12038644534488014684'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0106100357'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-06T15:53:56.134138Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-06_07_53_55-12038644534488014684]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-06_07_53_55-12038644534488014684
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-06_07_53_55-12038644534488014684?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-06_07_53_55-12038644534488014684 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:03.307Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.110Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.172Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.219Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.258Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.316Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.366Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.438Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.473Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.509Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.560Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.611Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.663Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.728Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.779Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:04.814Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:05.010Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:05.082Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:05.138Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:05.184Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-06_07_53_55-12038644534488014684 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:36.938Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:54:54.140Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:55:16.595Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T15:55:16.634Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:46.894Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:46.978Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:47.010Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:47.077Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:47.138Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:47.230Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T16:01:47.299Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T20:00:34.599Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-06_07_53_55-12038644534488014684.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T20:00:34.635Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T20:00:34.778Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T20:00:34.844Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-06T20:00:34.872Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-06_07_53_55-12038644534488014684 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 7m 50s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/patjnx3cups6q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #456
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/456/display/redirect?page=changes>
Changes:
[johnjcasey] [BEAM-12391] update avro sink to close the opened file handle, instead
[Robert Bradshaw] [BEAM-13482] Python fully qualified name external transforms.
[Robert Bradshaw] Add Python expansion service entry point.
[Kyle Weaver] [BEAM-13569] Change Spark dependencies to implementation.
[Kyle Weaver] remove redundant dependency
[wuren] [BEAM-13591] Bump log4j2 version to 2.17.1
[Steve Niemitz] [BEAM-13459] Update CHANGES.md, add note about artifact caching for
[relax] Add Flink runner support for OrderedListState. This version reads the
[noreply] Merge pull request #16404: [BEAM-13586] Fix NPE in
[zyichi] Fix sdk_container_builder too many values to unpack error
[noreply] [BEAM-13480] Sickbay PubSubIntegrationTest.test_streaming_data_only on
[Kyle Weaver] remove redundant testImplementation dependencies
[noreply] [BEAM-13430] Swap to use "mainClass" instead of "main" since it was
[zyichi] Fix remaining failing perf IT tests.
[noreply] [BEAM-13430] Replace deprecated "appendix" with "archiveAppendix"
[noreply] [BEAM-13015] Add jamm as a java agent to the Java SDK harness container
[noreply] [BEAM-13430] Partially revert
[noreply] Merge pull request #15863 from [BEAM-13184] Autosharding for
[noreply] [BEAM-11936] Enable FloatingPointAssertionWithinEpsilon errorprone check
[noreply] [BEAM-11936] Enable LockNotBeforeTry errorprone check (#16259)
[noreply] [BEAM-11936] Enable errorprone unused checks (#16262)
[noreply] Add Nexmark Query 14 (#16337)
[noreply] [BEAM-13015] Migrate all user state and side implementations to support
[noreply] [BEAM-13015] Use 20% of memory when the maximum has been configured.
------------------------------------------
[...truncated 374.47 KB...]
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.28)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.28 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.28)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.10)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2639683 sha256=152c3f9ae4aa148c926bfeb662807c4182bdd360ddd356366d6c069448bfa582
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6c62f3db00> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6c62ec1320> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0105100400.1641399879.168817/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0105100400.1641399879.168817/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0105100400.1641399879.168817/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0105100400.1641399879.168817/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220105162439169804-2259'
createTime: '2022-01-05T16:24:40.401154Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-05_08_24_39-3245966039095730749'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0105100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-05T16:24:40.401154Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-05_08_24_39-3245966039095730749]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-05_08_24_39-3245966039095730749
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-05_08_24_39-3245966039095730749?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-05_08_24_39-3245966039095730749 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:44.805Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.859Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.888Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.903Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.927Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.957Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.972Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:45.993Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.055Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.070Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.094Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.118Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.142Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.159Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.198Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.223Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.330Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.380Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.414Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:24:46.436Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-05_08_24_39-3245966039095730749 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:25:05.767Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:25:34.628Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:25:56.849Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:25:56.877Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:15.961Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.011Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.048Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.099Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.138Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.180Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T16:32:16.241Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T20:00:25.614Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-05_08_24_39-3245966039095730749.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T20:00:25.642Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T20:00:25.794Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-05_08_24_39-3245966039095730749 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T20:00:25.901Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-05T20:00:25.940Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 36m 55s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/ko2ble267qn34
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #455
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/455/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13509] Stop sharing SQS client between readers of same source.
[mmack] [BEAM-13587] Attempt to load AWS region from default provider chain in
------------------------------------------
[...truncated 374.95 KB...]
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.27.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.27)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.27 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.27)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=dbf4b50dbe8c66b41e738dc9ef74086aa7da7029e4913974bd02317e1efe1e55
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fbe7b00f680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fbe7b00fe60> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0104100356.1641313751.566375/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0104100356.1641313751.566375/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0104100356.1641313751.566375/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0104100356.1641313751.566375/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220104162911567447-3919'
createTime: '2022-01-04T16:29:12.836098Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-04_08_29_12-15028370625266252225'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0104100356'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-04T16:29:12.836098Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-04_08_29_12-15028370625266252225]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-04_08_29_12-15028370625266252225
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-04_08_29_12-15028370625266252225?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-04_08_29_12-15028370625266252225 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:16.969Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.666Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.706Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.733Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.766Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.808Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.840Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.864Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.889Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.922Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.956Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:17.989Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.062Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.089Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.113Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.140Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.273Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.337Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.383Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:18.414Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-04_08_29_12-15028370625266252225 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:29:46.198Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:30:02.940Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:30:26.985Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:30:27.003Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.731Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.799Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.821Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.876Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.922Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:44.985Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T16:36:45.031Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-04_08_29_12-15028370625266252225 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T20:00:43.901Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-04_08_29_12-15028370625266252225.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T20:00:43.929Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T20:00:44.060Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T20:00:44.237Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-04T20:00:44.267Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 32m 40s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/osvwru6zqjwpm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #454
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/454/display/redirect>
Changes:
------------------------------------------
[...truncated 376.12 KB...]
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.26)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.26)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=7cf78dc24e346f0a1e29740a0c6b97a8a48128db948776b55a55e02233a66170
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f90463e7560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f90463e7d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0103100355.1641227121.570476/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0103100355.1641227121.570476/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0103100355.1641227121.570476/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0103100355.1641227121.570476/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220103162521571465-2078'
createTime: '2022-01-03T16:25:22.855610Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-03_08_25_22-15494756019112572819'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0103100355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-03T16:25:22.855610Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-03_08_25_22-15494756019112572819]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-03_08_25_22-15494756019112572819
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-03_08_25_22-15494756019112572819?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-03_08_25_22-15494756019112572819 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:26.990Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.798Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.832Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.884Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.909Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.954Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:27.980Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.014Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.040Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.067Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.084Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.113Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.148Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.180Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.203Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.228Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-03_08_25_22-15494756019112572819 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.349Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.407Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.442Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:28.469Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:25:56.832Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:26:11.909Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:26:34.978Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:26:35.017Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.363Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.439Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.472Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.528Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.596Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.660Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T16:32:50.757Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-03_08_25_22-15494756019112572819 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T20:00:36.951Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-03_08_25_22-15494756019112572819.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T20:00:36.976Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T20:00:37.201Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T20:00:37.292Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T20:00:37.312Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 36m 20s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/fddl3iehcahy6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #453
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/453/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13498, BEAM-13522] Sickbay broken test and test category for Spark
------------------------------------------
[...truncated 375.87 KB...]
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.26)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=f0553c2493514fef189dfbcd39aef0324429fd25a0d9669ee9d47168a67c94fe
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f35124d3560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f35124d3d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0102135252.1641149213.315680/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0102135252.1641149213.315680/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0102135252.1641149213.315680/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0102135252.1641149213.315680/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220102184653316623-4296'
createTime: '2022-01-02T18:46:54.504789Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-02_10_46_53-4351009380020189362'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0102135252'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-02T18:46:54.504789Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-02_10_46_53-4351009380020189362]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-02_10_46_53-4351009380020189362
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-02_10_46_53-4351009380020189362?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-02_10_46_53-4351009380020189362 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:58.667Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.437Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.467Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.493Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.526Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.553Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.584Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.608Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.634Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.665Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.688Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.717Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.753Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.783Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.817Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:46:59.841Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:00.013Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:00.079Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:00.118Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:00.147Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-02_10_46_53-4351009380020189362 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:16.685Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:47:51.098Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:48:12.998Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:48:13.034Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.377Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.422Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.444Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.494Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.553Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.599Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-02T18:54:26.647Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-02_10_46_53-4351009380020189362 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T00:00:58.251Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-02_10_46_53-4351009380020189362.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T00:00:58.285Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T00:00:58.449Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T00:00:58.513Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-03T00:00:58.539Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 15m 9s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/55ssqudcyboyo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #452
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/452/display/redirect?page=changes>
Changes:
[stranniknm] [BEAM-13555] playground - quick fix for safari
[noreply] Merge pull request #15769 from [BEAM-13031] [Playground] Code editor -
[noreply] [BEAM-13541] Fix spotbugs issue with DLS in tail (#16407)
[noreply] [BEAM-13588] Add missing module for PVR tests and also sickbay known
[noreply] [BEAM-13575] Sickbay test that is flaky to restore precommit test signal
[noreply] [BEAM-12092] Bump jedis to version 3.8.0 (#16403)
------------------------------------------
[...truncated 375.19 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.26)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=154dbecd6b66f1131ffbec5d1e761735c771ac137ebcd341a1b178bd5fe6baf0
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fbe3b7d6560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fbe3b7d6d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0101100359.1641053749.574679/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0101100359.1641053749.574679/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0101100359.1641053749.574679/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-0101100359.1641053749.574679/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220101161549575694-8630'
createTime: '2022-01-01T16:15:50.746282Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-01_08_15_50-17882184911438861131'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-0101100359'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-01T16:15:50.746282Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-01_08_15_50-17882184911438861131]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-01_08_15_50-17882184911438861131
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-01_08_15_50-17882184911438861131?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-01_08_15_50-17882184911438861131 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:55.164Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:55.942Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:55.985Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.018Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.052Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.093Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.149Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.176Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.209Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.240Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.273Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.308Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.344Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.375Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.408Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.442Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.578Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.633Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.700Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:15:56.753Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-01_08_15_50-17882184911438861131 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:16:11.252Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:16:41.641Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:17:03.937Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:17:03.963Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.446Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.532Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.564Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.633Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.684Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.744Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T16:23:23.802Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T20:00:31.310Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-01-01_08_15_50-17882184911438861131.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T20:00:31.338Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T20:00:31.482Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T20:00:31.584Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-01T20:00:31.635Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-01_08_15_50-17882184911438861131 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 45m 51s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/apu3jtwhnseaa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #451
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/451/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13009] Fix tests to ensure absence of duplicates only per request
[mmack] [adhoc] Use predefined min/max combiners in SQS reader to gather stats
[zyichi] Fix class not found in perf integration tests.
[noreply] Update
[noreply] [BEAM-13430] Fix class not found error for example integration tests and
[noreply] [BEAM-13430] Re-enable dependency analysis for modules. (#16395)
[noreply] [BEAM-13484][Playground] Improve Terraform Scripts For Deploy
[noreply] [BEAM-13541] More intelligent caching of CoGBK values. (#16354)
------------------------------------------
[...truncated 375.81 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.26)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=ec16327bcc18eb26d7ee577bd9c0affa8c2d2b2f5f5cd4e880bd9248f2ffeb1d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7febc9858560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7febc9858d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1231100352.1640966710.315444/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1231100352.1640966710.315444/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1231100352.1640966710.315444/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1231100352.1640966710.315444/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211231160510316388-8513'
createTime: '2021-12-31T16:05:11.402272Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-31_08_05_10-15982918373833551312'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1231100352'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-31T16:05:11.402272Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-31_08_05_10-15982918373833551312]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-31_08_05_10-15982918373833551312
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-31_08_05_10-15982918373833551312?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-31_08_05_10-15982918373833551312 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.234Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.839Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.871Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.900Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.930Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.969Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:15.994Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.028Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.072Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.097Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.119Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.147Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.187Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.207Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.231Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.263Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.382Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.438Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-31_08_05_10-15982918373833551312 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.474Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:16.495Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:05:37.125Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:06:16.029Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:06:42.249Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:06:42.276Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:07.879Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:07.937Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:07.971Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:08.027Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:08.111Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:08.180Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T16:13:08.249Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T20:00:32.058Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-31_08_05_10-15982918373833551312.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T20:00:32.089Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T20:00:32.236Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T20:00:32.432Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-31T20:00:32.451Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-31_08_05_10-15982918373833551312 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 56m 30s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/xm6jhskmykiiw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #450
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/450/display/redirect?page=changes>
Changes:
[Steve Niemitz] [BEAM-13488] Work around bug in classpath detection
[noreply] [BEAM-13578] Validate that repeatable PipelineOptions need to be
[noreply] [BEAM-13430] Update comment over eclipse plugin. (#16387)
[zyichi] [BEAM-13430] Fix broken ULR validates runner tests
[noreply] Update venv creation commands in release scripts. (#16381)
[Steve Niemitz] [BEAM-13459] Cache artifacts across job runs on python+dataflow
[noreply] [BEAM-12777] Stable URL for current API docs (#16327)
[noreply] [BEAM-13539] correct resource override dependencies (#16392)
[Kyle Weaver] [BEAM-13581] Remove previous job name for Flink PVR precommit.
[emilyye] Moving to 2.37.0-SNAPSHOT on master branch.
[noreply] Release website update for Beam 2.35.0 (#16115)
[noreply] [BEAM-13430] Re-enable checkerframework for the project excluding the
------------------------------------------
[...truncated 375.23 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.3)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.20.26)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.37.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (1.23.26)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.37.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.37.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.2.0)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.37.0.dev0) (0.12.3)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (1.43.0)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (6.1.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.37.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.37.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.37.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.37.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (21.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (0.2.5)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.37.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.37.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.37.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.37.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.37.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.37.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.37.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2634775 sha256=d1dfb9b809ce290accddd1b07b8e8d751c090f8030dd4eee869b7000b6d494cf
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.37.0.dev0
Uninstalling apache-beam-2.37.0.dev0:
Successfully uninstalled apache-beam-2.37.0.dev0
Successfully installed apache-beam-2.37.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0eb300a560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0eb300ad40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1230100341.1640880188.744302/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1230100341.1640880188.744302/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1230100341.1640880188.744302/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1230100341.1640880188.744302/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211230160308745271-7569'
createTime: '2021-12-30T16:03:09.808782Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-30_08_03_09-5035936667533043741'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1230100341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-30T16:03:09.808782Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-30_08_03_09-5035936667533043741]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-30_08_03_09-5035936667533043741
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-30_08_03_09-5035936667533043741?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-30_08_03_09-5035936667533043741 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:14.098Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:14.921Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:14.959Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:14.985Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.042Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.079Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.112Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.146Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.177Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.210Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.241Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.273Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.307Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.359Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.412Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.445Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.594Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.658Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.705Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:15.734Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-30_08_03_09-5035936667533043741 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:03:29.308Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:04:06.310Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:04:27.643Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:04:27.670Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.478Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.557Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.592Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.677Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.732Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.821Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T16:10:42.906Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T20:00:40.037Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-30_08_03_09-5035936667533043741.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T20:00:40.063Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T20:00:40.185Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T20:00:40.290Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-30T20:00:40.311Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-30_08_03_09-5035936667533043741 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1631, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 58m 40s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/ke2keyywyyacm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #449
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/449/display/redirect?page=changes>
Changes:
[dpcollins] [BEAM-13402] Simplify PubsubLiteSink
[Kyle Weaver] [BEAM-13521] flink pvr batch precommit
[Kyle Weaver] [BEAM-13571] Fix ClassNotFound exception in Flink tests
[Kyle Weaver] [BEAM-13498] [BEAM-13573] exclude new tests on Flink
[noreply] Exclude UsesOnWindowExpiration by category from Dataflow v2 streaming
[noreply] [BEAM-13052] Increment pubsub python version and fix breakages. (#16126)
[noreply] [BEAM-13052] Add Pub/Sub Lite xlang transforms in python (#15727)
[noreply] [BEAM-13402] Version bump Pub/Sub Lite and implement changes to ensure
------------------------------------------
[...truncated 375.19 KB...]
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<3,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.0)
Requirement already satisfied: google-cloud-pubsublite<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.2.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.1)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: overrides<7.0.0,>=6.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.36.0.dev0) (6.1.0)
Requirement already satisfied: grpcio-status>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.3.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: typing-utils>=0.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from overrides<7.0.0,>=6.0.1->google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.36.0.dev0) (0.1.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2632854 sha256=3c80cf2e9959803ebdcc4f9c632e029446a35a13c1dd6f73931ffd3dbb8ac037
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2b5c21ad40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2b5c216560> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1229100355.1640793134.452259/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1229100355.1640793134.452259/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1229100355.1640793134.452259/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1229100355.1640793134.452259/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211229155214453379-5169'
createTime: '2021-12-29T15:52:15.670803Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-29_07_52_14-968500281168903203'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1229100355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-29T15:52:15.670803Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-29_07_52_14-968500281168903203]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-29_07_52_14-968500281168903203
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-29_07_52_14-968500281168903203?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-29_07_52_14-968500281168903203 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:20.360Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.760Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.803Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.835Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.859Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.889Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.921Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.957Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:21.992Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.062Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.096Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.126Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.155Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.189Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.214Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.350Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.421Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.466Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:22.500Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-29_07_52_14-968500281168903203 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:52:41.426Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:53:13.070Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:53:35.949Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:53:35.977Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.110Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.194Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.233Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.308Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.368Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.444Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T15:59:56.523Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-29_07_52_14-968500281168903203 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T20:00:36.669Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-29_07_52_14-968500281168903203.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T20:00:36.699Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T20:00:36.889Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T20:00:36.943Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-29T20:00:36.963Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 9m 31s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/vxvvwfapgvtbg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #448
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/448/display/redirect?page=changes>
Changes:
[mmack] [adhoc] Forbid to import guava and others from org.testcontainers.shaded
[noreply] [BEAM-13526] Kafka.IO: make DeserializerProvider a public interface
------------------------------------------
[...truncated 369.06 KB...]
> Task :buildSrc:build UP-TO-DATE
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628595 sha256=151e9db82816844077c158ded8d1c4bdcadf4da4a09779066f0d9147594eafe0
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f45a27e7050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f45a27e7830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1228100337.1640707254.122978/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1228100337.1640707254.122978/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1228100337.1640707254.122978/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1228100337.1640707254.122978/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211228160054123979-3971'
createTime: '2021-12-28T16:00:55.531938Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-28_08_00_54-7043686246828525726'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1228100337'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-28T16:00:55.531938Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-28_08_00_54-7043686246828525726]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-28_08_00_54-7043686246828525726
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-28_08_00_54-7043686246828525726?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-28_08_00_54-7043686246828525726 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:00:59.519Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.744Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.781Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.807Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.834Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.861Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.889Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.914Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.934Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:00.988Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.034Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.059Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.081Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.100Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.150Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.186Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.316Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.380Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.428Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:01.457Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-28_08_00_54-7043686246828525726 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:27.886Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:01:46.563Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:02:10.426Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:02:10.483Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:29.724Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:29.800Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:29.831Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:29.901Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:29.953Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:30.011Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T16:08:30.082Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T20:00:32.824Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-28_08_00_54-7043686246828525726.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T20:00:32.851Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T20:00:32.977Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T20:00:33.040Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-28T20:00:33.057Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-28_08_00_54-7043686246828525726 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 46s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/yrwxavtso4cd2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #447
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/447/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16285 from [BEAM-13492][Playground] Update backend
------------------------------------------
[...truncated 367.40 KB...]
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.8)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628595 sha256=e5826097932a16e2e1672566cf1ff299e479cee08b3d159c80b1a7dc5aafed98
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa0af7b2050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa0af7b2830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1227100334.1640620682.840980/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1227100334.1640620682.840980/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1227100334.1640620682.840980/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1227100334.1640620682.840980/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211227155802841971-1448'
createTime: '2021-12-27T15:58:04.016331Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-27_07_58_03-537830629594930732'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1227100334'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-27T15:58:04.016331Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-27_07_58_03-537830629594930732]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-27_07_58_03-537830629594930732
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-27_07_58_03-537830629594930732?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-27_07_58_03-537830629594930732 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:08.438Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.050Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.096Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.128Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.170Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.207Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.242Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.264Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.297Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.335Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.367Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.405Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.444Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.480Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.506Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.536Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.671Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.746Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.795Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:09.819Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-27_07_58_03-537830629594930732 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:33.570Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:58:48.974Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:59:12.779Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T15:59:12.840Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.167Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.265Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.298Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.367Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.444Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.504Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T16:05:31.565Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T20:00:33.573Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-27_07_58_03-537830629594930732.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T20:00:33.599Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T20:00:33.711Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T20:00:33.833Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-27T20:00:33.877Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-27_07_58_03-537830629594930732 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 41s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/4pfm7z4fzuro2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #446
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/446/display/redirect?page=changes>
Changes:
[alexander.zhuravlev] [BEAM-13530] Fixed scrollbar in the Output section
[noreply] Merge pull request #16355 from [BEAM-13529] [Playground] [Bugfix] Fix
[noreply] Fixed grammar error in Go ParDo function comments. (#16329)
------------------------------------------
[...truncated 367.55 KB...]
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628595 sha256=8cb52675e68a9444fbf22e8163ed49c94f7edaced451ce2ced275f23a23b9832
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f094a58d050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f094a58d830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1226100339.1640534575.523330/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1226100339.1640534575.523330/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1226100339.1640534575.523330/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1226100339.1640534575.523330/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211226160255524291-7386'
createTime: '2021-12-26T16:02:56.750483Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-26_08_02_56-6233636307437352158'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1226100339'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-26T16:02:56.750483Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-26_08_02_56-6233636307437352158]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-26_08_02_56-6233636307437352158
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-26_08_02_56-6233636307437352158?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-26_08_02_56-6233636307437352158 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:00.538Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.250Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.287Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.320Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.351Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.389Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.418Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.451Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.484Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.516Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.539Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.569Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.593Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.615Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.640Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.674Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.804Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-26_08_02_56-6233636307437352158 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.873Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.920Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:01.953Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:14.302Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:03:47.307Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:04:12.908Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:04:12.934Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.011Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.083Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.116Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.182Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.231Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.309Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T16:10:41.358Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T20:00:33.512Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-26_08_02_56-6233636307437352158.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T20:00:33.538Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T20:00:33.651Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T20:00:33.775Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-26T20:00:33.797Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-26_08_02_56-6233636307437352158 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 58m 45s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/l7c326b4343ky
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #445
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/445/display/redirect>
Changes:
------------------------------------------
[...truncated 366.69 KB...]
> Task :buildSrc:build UP-TO-DATE
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628595 sha256=009bed557a855c61c97a6e39c16feb73887510f037d98e873d5d2fbd0e7e1cb5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6e23a69050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6e23a69830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1225100353.1640448263.329417/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1225100353.1640448263.329417/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1225100353.1640448263.329417/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1225100353.1640448263.329417/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211225160423330438-2297'
createTime: '2021-12-25T16:04:24.391253Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-25_08_04_23-16334911706583071641'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1225100353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-25T16:04:24.391253Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-25_08_04_23-16334911706583071641]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-25_08_04_23-16334911706583071641
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-25_08_04_23-16334911706583071641?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-25_08_04_23-16334911706583071641 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:28.629Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.524Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.565Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.584Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.613Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.653Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.673Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.712Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.734Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.757Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.786Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.819Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.851Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.874Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.907Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:29.937Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:30.121Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:30.217Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:30.262Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:04:30.280Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-25_08_04_23-16334911706583071641 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:05:00.866Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:05:17.388Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:05:38.356Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:05:38.387Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.328Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.394Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.417Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.460Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.505Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.551Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T16:11:47.608Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T20:00:29.445Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-25_08_04_23-16334911706583071641.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T20:00:29.473Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T20:00:29.658Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T20:00:29.696Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-25T20:00:29.719Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-25_08_04_23-16334911706583071641 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 57m 15s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/5ajb6mln2xkbi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #444
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/444/display/redirect?page=changes>
Changes:
[alexander.zhuravlev] [BEAM-13489] Updated manifest.json and index.html to follow best
[stranniknm] [BEAM-13531] playground - fetch example info only once
[mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v2)
[mmack] [BEAM-7559] Add integration test for DynamoDBIO (SDK v1)
[noreply] [BEAM-13399] Disable expansion service integration test on Samza and
[noreply] [BEAM-13430] Upgrade beam to gradle 7.3.2 (#16319)
[noreply] Merge pull request #16320 from [BEAM-13512] [Playground] send request to
[noreply] Merge pull request #16209 from [Playground][BEAM-13304]Deploy Python
[noreply] Merge pull request #16105 from [BEAM-13328] [Playground] Add
[noreply] Merge pull request #16103 from [BEAM-13252] [Playground] Feedback Thumbs
[noreply] Better type inference for GroupBy. (#16318)
[noreply] [BEAM-13534] Update TestAutomatedExpansionService liveness check to not
[noreply] [BEAM-13490] Bump log4j version to 2.17.0 (#16276)
------------------------------------------
[...truncated 367.42 KB...]
> Task :buildSrc:build UP-TO-DATE
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.1.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628595 sha256=fabcf137673606e301b36d2b0c6e0e7d3d1078d0b4f0ad38369561fe70fd3cbe
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe6f396c050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe6f396c830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1224100356.1640360926.732776/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1224100356.1640360926.732776/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1224100356.1640360926.732776/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1224100356.1640360926.732776/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211224154846733739-3597'
createTime: '2021-12-24T15:48:47.887510Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-24_07_48_47-17157577602805147684'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1224100356'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-24T15:48:47.887510Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-24_07_48_47-17157577602805147684]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-24_07_48_47-17157577602805147684
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-24_07_48_47-17157577602805147684?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-24_07_48_47-17157577602805147684 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:51.735Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.531Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.570Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.589Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.615Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.639Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.667Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.701Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.748Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.794Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.822Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.841Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.879Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.906Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.939Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:52.971Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:53.101Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:53.173Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:53.205Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:48:53.249Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-24_07_48_47-17157577602805147684 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:49:19.326Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:49:40.173Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:50:03.017Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:50:03.057Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:35.958Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.026Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.061Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.200Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.268Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.356Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T15:56:36.407Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T20:00:21.629Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-24_07_48_47-17157577602805147684.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T20:00:21.663Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T20:00:21.814Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T20:00:21.953Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-24T20:00:21.977Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-24_07_48_47-17157577602805147684 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 12m 46s
15 actionable tasks: 2 executed, 13 up-to-date
Publishing build scan...
https://gradle.com/s/na3deryi3s2ea
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #443
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/443/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13494] Avoid draining Kinesis API limits & CPU when reading from
[arietis27] [BEAM-13517] Unable to write nulls to columns with logical types
[stranniknm] [BEAM-13501] playground - add log about processing and precompiled run
[noreply] Make S3 streaming more efficient (#15931)
[noreply] [Playground][BEAM-13358]Deploy Go service via Ci/CD (#16210)
[dpcollins] [BEAM-13430] Clean up tests that override the time of the JVM
[noreply] [BEAM-12830] Install go version w/host platform. (#16330)
[noreply] [BEAM-13399] Add extra time buffer to reduce likelihood of flake in
[noreply] [BEAM-12865] Fix triggering_frequency validation (#16315)
[noreply] [BEAM-13525] Sickbay new PardoTests for Dataflow V2 Java Streaming VR
[noreply] [BEAM-12830] Avoid flock and prepare task once. (OSX fix.) (#16333)
[noreply] Bump Python beam-master containers (#16332)
[arietis27] [BEAM-13517] Unable to write nulls to columns with logical types
------------------------------------------
[...truncated 366.26 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.29)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.0.4)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2628357 sha256=53ab527313d24edcc4cb1be0f85aab91a7c32737703a8e26d3bf87638dbb990d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211222" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7c08e76ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7c08e79710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1223100415.1640275263.511712/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1223100415.1640275263.511712/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1223100415.1640275263.511712/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1223100415.1640275263.511712/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211223160103512719-3616'
createTime: '2021-12-23T16:01:07.500844Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-23_08_01_03-3478904842905076970'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1223100415'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-23T16:01:07.500844Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-23_08_01_03-3478904842905076970]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-23_08_01_03-3478904842905076970
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-23_08_01_03-3478904842905076970?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-23_08_01_03-3478904842905076970 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:14.895Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.754Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.805Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.831Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.853Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.885Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.921Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:15.993Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.072Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.152Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.193Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.226Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.264Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.299Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.332Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.382Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.508Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.586Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.625Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:16.658Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-23_08_01_03-3478904842905076970 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:25.658Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:01:59.796Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:02:23.523Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:02:23.545Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.549Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.608Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.643Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.707Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.761Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.828Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T16:08:36.897Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T20:00:40.981Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-23_08_01_03-3478904842905076970.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T20:00:41.006Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T20:00:41.137Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T20:00:41.184Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-23T20:00:41.201Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-23_08_01_03-3478904842905076970 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 0m 43s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/u4hzbpi32g4fk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #442
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/442/display/redirect?page=changes>
Changes:
[rsl4] [BEAM-13397] Bump numpy to 1.21 for M1 chip
[rsl4] re-generate requirements.txt
[Kyle Weaver] [BEAM-13498] Sickbay portable Flink testProcessElementSkew.
[stranniknm] [BEAM-13502]: fix loading example on embedded version
[noreply] [BEAM-13430] Start the process of upgrading the Gradle 7. (#16292)
[noreply] [BEAM-13430] Introduce new testRuntimeMigration configuration that
[noreply] Update flink cluster to use a supported dataproc version (1.2 -> 1.5)
[Robert Bradshaw] Add microbenchmark for row coder.
[noreply] [BEAM-13499] Add warning about hcatalog to release notes and javadoc
[Robert Bradshaw] Compile RowCoderImpl with Cython.
[Robert Bradshaw] RowCoder optimizations.
[Robert Bradshaw] Further optimizations for common case of components in order.
[noreply] [BEAM-13430] Remove propdeps and replace with compileOnly (#16308)
[Robert Bradshaw] Optimize rows with null fields.
[noreply] Merge pull request #16304 from [BEAM-13491] [Playground] Examples'
[noreply] Merge pull request #16283 from [BEAM-13448] [Playground] track run code
[noreply] Merge pull request #16244 from [BEAM-13463] [Playground] add retries to
[noreply] Merge pull request #16241 from [BEAM-13440] [Playground] Implement
[noreply] [BEAM-13399] Add integration test for Go SDK expansion service JAR
[noreply] [BEAM-13421] Fix bug with xs called with non-tuple key (#16258)
[Robert Bradshaw] RowCoder lint.
------------------------------------------
[...truncated 366.16 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.22.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.5)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.26)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.26 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.26)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.0.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2627895 sha256=7d784a29b9bfc233969c7007d381120b631bb2c2db3518125f4db073c92e8e0a
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc6066badd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc6066bd5f0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1222100351.1640188581.446872/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1222100351.1640188581.446872/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1222100351.1640188581.446872/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1222100351.1640188581.446872/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211222155621447830-7859'
createTime: '2021-12-22T15:56:22.675827Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-22_07_56_21-11667092084609528145'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1222100351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-22T15:56:22.675827Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-22_07_56_21-11667092084609528145]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-22_07_56_21-11667092084609528145
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-22_07_56_21-11667092084609528145?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-22_07_56_21-11667092084609528145 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:28.686Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.487Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.538Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.569Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.602Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.645Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.679Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.705Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.734Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.767Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.795Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.838Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.867Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.898Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.931Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:29.987Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:30.160Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:30.213Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:30.250Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:30.281Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-22_07_56_21-11667092084609528145 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:56:49.674Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:57:18.811Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:57:41.847Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T15:57:41.881Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:00.934Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.016Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.048Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.115Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.159Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.262Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T16:04:01.337Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-22_07_56_21-11667092084609528145 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T20:00:33.598Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-22_07_56_21-11667092084609528145.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T20:00:33.625Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T20:00:33.787Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T20:00:33.938Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-22T20:00:34.031Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 5m 20s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/n6wiuykwdb4mw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #441
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/441/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-13088] Add parameter tempWriteDataset to BigQueryIO to store temp
[stranniknm] [BEAM-13466]: sort categories and examples by name
[aydar.zaynutdinov] [BEAM-13485][Playground]
[alexander.zhuravlev] [BEAM-13476] Changed Timeout Error Text
[alexander.zhuravlev] [BEAM-13474] Changed 'Playground' logo text color
[mmack] [BEAM-13443] Avoid blocking put to Kinesis records queue to shutdown
[noreply] Merge pull request #16278 from [BEAM-13479] [Playground] Change logic
[noreply] Merge pull request #16281 from [BEAM-13475] [Playground] [Bugfix] Error
[noreply] Merge pull request #16279 from [BEAM-13473] [Playground] [Bugfix] Reset
[noreply] Merge pull request #16232 from [BEAM-13285][Playground] Add steps to
[noreply] [BEAM-13399] Add check for dev versions of JARs to download code
[noreply] Merge pull request #16122 from [BEAM-13345] [Playground] add resizable
[benjamin.gonzalez] [BEAM-13088] Make tempDataset final
[noreply] Merge pull request #16170 from [BEAM-13411][Playground] Add getting of
[noreply] Merge pull request #16172 from [BEAM-13417] [Playground] Add java
[noreply] Merge pull request #16240 from [BEAM-13465][Playground] Change error
[noreply] Merge pull request #16234 from [BEAM-13461][Playground] [Bugfix] Error
[noreply] Merge pull request #15489 from [BEAM-12865] Allow custom batch_duration
[noreply] [BEAM-13483] Increase timeout of Java Examples Dataflow suite. (#16226)
[Kyle Weaver] [BEAM-13496] Upgrade Flink runner to include log4j patches.
[Kyle Weaver] [BEAM-13497] Correct class name in Flink tests.
[noreply] Pin transitive log4j dependencies to 2.17.0 in :sdks:java:io:hcatalog
[noreply] [BEAM-13434] Bump solr to 8.11.1 (#16296)
[noreply] Use a patched shadow 6.1.0 plugin using Log4j 2.16.0 (#16269)
[noreply] [BEAM-12830] Replace GoGradle plugin with Shell Scripts. (#16291)
------------------------------------------
[...truncated 365.95 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.25)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.25 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.25)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.0.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2627274 sha256=3745ff235825d8adbbacc5c399f886a864c007396116efe710b91e02cff3ee00
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd7571a24d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd7571a2cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1221100356.1640102245.331020/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1221100356.1640102245.331020/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1221100356.1640102245.331020/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1221100356.1640102245.331020/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211221155725332070-1244'
createTime: '2021-12-21T15:57:26.491437Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-21_07_57_25-10194202764553797347'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1221100356'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-21T15:57:26.491437Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-21_07_57_25-10194202764553797347]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-21_07_57_25-10194202764553797347
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-21_07_57_25-10194202764553797347?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-21_07_57_25-10194202764553797347 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:31.369Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.423Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.523Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.566Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.599Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.671Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.715Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.748Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.782Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.817Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.840Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.875Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.912Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.951Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:32.985Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:33.025Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:33.234Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:33.307Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:33.361Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:33.391Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-21_07_57_25-10194202764553797347 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:57:43.012Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:58:16.977Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:58:39.906Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T15:58:39.935Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.228Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.327Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.372Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.450Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.508Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.584Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T16:04:55.651Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T20:00:22.793Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-21_07_57_25-10194202764553797347.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T20:00:22.824Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T20:00:22.967Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T20:00:23.133Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-21T20:00:23.182Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-21_07_57_25-10194202764553797347 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 5s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/h72hroybo52ko
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #440
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/440/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13233] Replace AWS API used to list shards from DescribeStream to
------------------------------------------
[...truncated 366.71 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (60.0.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2626967 sha256=948f2e02cbb7226f5cf349762148e4efd369e2ea33826164c3d1b50dfeeaf223
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb3d3af5560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb3d3af5d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1220100341.1640016238.024126/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1220100341.1640016238.024126/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1220100341.1640016238.024126/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1220100341.1640016238.024126/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211220160358025209-3086'
createTime: '2021-12-20T16:03:59.966101Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-20_08_03_58-9609965887796412227'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1220100341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-20T16:03:59.966101Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-20_08_03_58-9609965887796412227]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-20_08_03_58-9609965887796412227
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-20_08_03_58-9609965887796412227?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-20_08_03_58-9609965887796412227 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:04.489Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:05.478Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.423Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.606Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.741Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.803Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.838Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.868Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.897Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.920Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.942Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:06.977Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.016Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.046Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.082Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.106Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.242Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.307Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.353Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:07.383Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-20_08_03_58-9609965887796412227 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:32.993Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:04:52.843Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:05:18.375Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:05:18.402Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.532Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.632Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.664Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.722Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.795Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.868Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T16:11:34.919Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-20_08_03_58-9609965887796412227 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T20:00:32.073Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-20_08_03_58-9609965887796412227.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T20:00:32.095Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T20:00:32.216Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T20:00:32.309Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-20T20:00:32.334Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 57m 41s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/twez26gsuwlby
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #439
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/439/display/redirect>
Changes:
------------------------------------------
[...truncated 365.95 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.7.0)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2626967 sha256=a7105bf93ad4ad33c64cc5e1abad4852ee78ecde956634e3890f8b81ca79d1ca
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f65bed11560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f65bed11d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1219100340.1639929693.668746/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1219100340.1639929693.668746/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1219100340.1639929693.668746/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1219100340.1639929693.668746/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211219160133669886-7261'
createTime: '2021-12-19T16:01:34.796109Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-19_08_01_34-15143121793166488700'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1219100340'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-19T16:01:34.796109Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-19_08_01_34-15143121793166488700]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-19_08_01_34-15143121793166488700
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-19_08_01_34-15143121793166488700?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-19_08_01_34-15143121793166488700 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:39.289Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:39.984Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.032Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.075Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.108Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.155Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.180Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.213Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.246Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.279Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.313Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.341Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.385Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.420Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.447Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.467Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.609Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.687Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.734Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:01:40.775Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-19_08_01_34-15143121793166488700 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:02:09.214Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:02:32.452Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:02:54.694Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:02:54.725Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:11.701Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:11.744Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:11.782Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:11.945Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:12.009Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:12.057Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T16:09:12.109Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-19_08_01_34-15143121793166488700 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T20:00:34.757Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-19_08_01_34-15143121793166488700.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T20:00:34.783Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T20:00:35.858Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T20:00:35.928Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-19T20:00:35.946Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 0m 7s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/begt6low35z4g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #438
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/438/display/redirect>
Changes:
------------------------------------------
[...truncated 365.38 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.7.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2626967 sha256=c48b8d8d485d92e4411bd43db0f5bdb0fd992e936e5d1e58ed51f1b32619f596
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f337a4a8560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f337a4a8d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1218100339.1639843325.943476/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1218100339.1639843325.943476/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1218100339.1639843325.943476/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1218100339.1639843325.943476/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211218160205944615-9903'
createTime: '2021-12-18T16:02:07.212096Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-18_08_02_06-10504444144374231036'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1218100339'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-18T16:02:07.212096Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-18_08_02_06-10504444144374231036]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-18_08_02_06-10504444144374231036
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-18_08_02_06-10504444144374231036?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-18_08_02_06-10504444144374231036 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:50.864Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.634Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.706Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.751Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.784Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.828Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.860Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.892Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.924Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.947Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.971Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:51.999Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.031Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.064Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.115Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.147Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.289Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.346Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.381Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:02:52.414Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-18_08_02_06-10504444144374231036 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:03:17.862Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:03:41.941Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:04:06.867Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:04:06.891Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.628Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.722Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.791Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.857Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.914Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:28.984Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T16:10:29.045Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T20:00:31.175Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-18_08_02_06-10504444144374231036.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T20:00:31.202Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T20:00:31.311Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T20:00:31.401Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-18T20:00:31.427Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-18_08_02_06-10504444144374231036 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 59m 36s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/dcbql6nhb7goq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #437
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/437/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16061 from [BEAM-13428] [Playground] Integrate
[noreply] Clarify CoGroupByKey creates Iterable, not list. (#16099)
[noreply] [BEAM-12931] Allow for DoFn#getAllowedTimestampSkew() when checking the
[noreply] [BEAM-13467] Properly handle null argument types for logical types.
[noreply] [BEAM-10277] Initial implementation for encoding position in Python
[noreply] [BEAM-11545] State & timer for batched RPC calls pattern (#13643)
[noreply] Automatically prune local images before building an RC. (#16238)
[noreply] Add verbose error messages to container-related scripts. (#16056)
[noreply] [BEAM-13456] Rollback #15890 to fix timeout in Java PostCommit (#16257)
[noreply] [BEAM-13015] Add a state backed iterable that can be mutated under
[noreply] [BEAM-13388] Update Cloud DLP after breaking changes. (#16236)
[noreply] [BEAM-13434] Bump google pubsublite on master. (#16265)
[mmack] [adhoc] Fix guava imports in tests
------------------------------------------
[...truncated 365.59 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.43.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.6.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2626967 sha256=4f8f7440e0a69b9e537f7827393d947c43f48272f82aa11d6471f8663588535b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe019316560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe019316d40> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1217100410.1639758804.052428/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1217100410.1639758804.052428/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1217100410.1639758804.052428/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1217100410.1639758804.052428/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211217163324053671-4633'
createTime: '2021-12-17T16:33:25.126134Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-17_08_33_24-9023568561121125785'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1217100410'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-17T16:33:25.126134Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-17_08_33_24-9023568561121125785]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-17_08_33_24-9023568561121125785
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-17_08_33_24-9023568561121125785?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-17_08_33_24-9023568561121125785 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:31.794Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.699Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.770Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.800Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.827Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.874Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.907Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.955Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:32.979Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.014Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.048Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.065Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.109Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.157Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.194Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.239Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.415Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.482Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.544Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:33:33.581Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-17_08_33_24-9023568561121125785 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:34:02.927Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:34:19.914Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:34:43.193Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:34:43.224Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:56.829Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:56.892Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:56.913Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:57.374Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:57.680Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:58.176Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T16:40:58.649Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T20:00:23.999Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-17_08_33_24-9023568561121125785.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T20:00:24.033Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T20:00:24.213Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T20:00:24.307Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-17T20:00:24.337Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-17_08_33_24-9023568561121125785 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 28m 9s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/vlzodvsklx5cm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #436
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/436/display/redirect?page=changes>
Changes:
[thiagotnunes] [BEAM-12164] Add Spanner Change Stream DAOs
[mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success
[noreply] [BEAM-13341] Serialize reflection parameter in AvroCoder cloud object
[noreply] Update grafana from 8.1.6 to 8.1.8
[noreply] [BEAM-13015] Update FakeBeamFnStateClient to generate elements that stop
[noreply] [BEAM-13218] Sickbay
[noreply] [BEAM-13399] Add infrastructure to start JARs from Go functions (#16214)
------------------------------------------
[...truncated 365.97 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.6.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.5)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2625849 sha256=283d89f10c1a6a633fc03d7b2b475cc45b53541f0141e73f24bc05d668ef926a
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc3e5225200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc3e52259e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1216100336.1639670456.032427/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1216100336.1639670456.032427/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1216100336.1639670456.032427/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1216100336.1639670456.032427/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211216160056033505-4367'
createTime: '2021-12-16T16:00:57.229022Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-16_08_00_56-10672804900396782061'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1216100336'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-16T16:00:57.229022Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-16_08_00_56-10672804900396782061]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-16_08_00_56-10672804900396782061
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-16_08_00_56-10672804900396782061?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-16_08_00_56-10672804900396782061 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:01.354Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.188Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.225Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.251Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.275Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.312Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.337Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.370Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.420Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.446Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.473Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.497Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.525Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.560Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.594Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.625Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.792Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.850Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.887Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:02.923Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-16_08_00_56-10672804900396782061 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:20.456Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:01:52.731Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:02:17.387Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:02:17.443Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.143Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.219Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.241Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.328Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.377Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.447Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T16:08:41.519Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T20:00:31.400Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-16_08_00_56-10672804900396782061.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T20:00:31.425Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T20:00:31.595Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T20:00:31.715Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-16T20:00:31.731Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-16_08_00_56-10672804900396782061 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 0m 45s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/v5rsub2rp2qyo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #435
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/435/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13438][Playground]
[stranniknm] [BEAM-13446]: fix incorrect tab symbol
[noreply] Removes the comment that seems no longer relevant.
[Valentyn Tymofieiev] Update a few dependencies that may depend on log4j transitively.
[noreply] [BEAM-13159] Add Redis Stream (XADD) Write Support (#15858)
[noreply] Merge pull request #15994: [BEAM-13263] Support OnWindowExpiration in
[noreply] Bump dataflow container version to beam-master-20211213 (#16213)
[noreply] Merge pull request #16198 from [BEAM-13437][Playground] Add
[noreply] Merge pull request #16195 from a[BEAM-13436][Playground] Add
[noreply] [BEAM-13355] add Big Query parameter to enable users to specify load_…
[noreply] Updated the base images to use debian:bullseye (#16221)
[noreply] [BEAM-13434] Bump log4j to 2.16.0. (#16237)
[mmack] [BEAM-13209] Fix DynamoDBIO.write to properly handle partial success
------------------------------------------
[...truncated 366.88 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.1)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.6.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2625824 sha256=a5faa487cbf18cf8a73638529f88cb80a25063a9db084790cecb924a8ac47fe5
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211213" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f89d24a3200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f89d24a39e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1215100338.1639585146.406965/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1215100338.1639585146.406965/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1215100338.1639585146.406965/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1215100338.1639585146.406965/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211215161906408005-3127'
createTime: '2021-12-15T16:19:07.503127Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-15_08_19_06-17670965029545745321'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1215100338'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-15T16:19:07.503127Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-15_08_19_06-17670965029545745321]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-15_08_19_06-17670965029545745321
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-15_08_19_06-17670965029545745321?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-15_08_19_06-17670965029545745321 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:11.611Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.571Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.601Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.632Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.658Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.722Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.747Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.781Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.812Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.873Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.900Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.931Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.962Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:12.989Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.024Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.047Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.195Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.261Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.302Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:13.333Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-15_08_19_06-17670965029545745321 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:22.765Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:19:56.891Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:20:22.284Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:20:22.320Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.142Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.227Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.271Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.339Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.403Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.463Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T16:26:44.532Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T20:00:27.138Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-15_08_19_06-17670965029545745321.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T20:00:27.167Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T20:00:27.301Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T20:00:27.406Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-15T20:00:27.432Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-15_08_19_06-17670965029545745321 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 42m 30s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/sc3ecowc5ubsy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #434
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/434/display/redirect?page=changes>
Changes:
[Daniel Oliveira] [BEAM-13321] Pass TempLocation as pipeline option to Dataflow Go for
[Robert Bradshaw] Better type hints for Count combiners.
[Kyle Weaver] Include name of missing tag in error message.
[stranniknm] [BEAM-13423]: fix frontend failure if no examples
[daria.malkova] change return type of 2 methods
[mmack] [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only
[noreply] Updating Grafana from v8.1.2 to v8.1.6
[daria.malkova] Docs for validators tests
[daria.malkova] change context type
[noreply] Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD
[noreply] Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs
[noreply] Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support
[mmack] [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and
[noreply] Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to
[noreply] Merge pull request #16179 from [BEAM-13344][Playground] support python
[noreply] Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log
[noreply] [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062)
[noreply] Change Pub/Sub Lite PollResult to set explicit watermark (#16216)
[noreply] [BEAM-13454] Fix and test dataframe read_fwf. (#16064)
[noreply] [BEAM-12976] Pipeline visitor to discover pushdown opportunities.
[noreply] [BEAM-13015] Allow decoding a set of elements until we hit the block
------------------------------------------
[...truncated 365.97 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.24)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.24 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.24)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.6.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.1)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2625414 sha256=75b96a5a3bc4db16982a775bd6784049af969055997eb2f3669b49c3d3e4a00c
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f35f4320290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f35f4320a70> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1214100356.1639498197.862245/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1214100356.1639498197.862245/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1214100356.1639498197.862245/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1214100356.1639498197.862245/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211214160957863611-3007'
createTime: '2021-12-14T16:09:59.021627Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-14_08_09_58-10373148539541120327'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1214100356'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-14T16:09:59.021627Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-14_08_09_58-10373148539541120327]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-14_08_09_58-10373148539541120327
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-14_08_09_58-10373148539541120327?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-14_08_09_58-10373148539541120327 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:05.763Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.312Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.437Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.492Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.552Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.634Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.666Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.710Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.755Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.798Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.843Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.877Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:07.924Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.005Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.031Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.074Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.409Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.611Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.727Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:08.784Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-14_08_09_58-10373148539541120327 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:38.415Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:10:55.481Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:11:21.562Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:11:21.604Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:14.933Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.055Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.092Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.181Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.518Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.600Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T16:17:15.681Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T20:00:29.512Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-14_08_09_58-10373148539541120327.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T20:00:29.561Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T20:00:29.738Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T20:00:29.845Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-14T20:00:29.884Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-14_08_09_58-10373148539541120327 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 51m 40s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/mfl7yfrzfwjje
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #433
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/433/display/redirect>
Changes:
------------------------------------------
[...truncated 366.45 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.5)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.23)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.23 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.23)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.6.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2625001 sha256=5fb69ff4a0f1a996767976fe8da6419b242d22d566e2fdc15cde3cbfab74d135
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f027a7a30e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f027a7a38c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1213100400.1639412681.733745/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1213100400.1639412681.733745/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1213100400.1639412681.733745/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1213100400.1639412681.733745/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211213162441734764-6798'
createTime: '2021-12-13T16:24:42.886452Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-13_08_24_42-3469144068838265839'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1213100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-13T16:24:42.886452Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-13_08_24_42-3469144068838265839]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-13_08_24_42-3469144068838265839
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-13_08_24_42-3469144068838265839?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-13_08_24_42-3469144068838265839 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:48.235Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.336Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.378Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.405Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.444Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.483Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.515Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.551Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.583Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.621Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.658Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.689Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.717Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.745Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.780Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:49.812Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:50.001Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:50.058Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:50.098Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:24:50.128Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-13_08_24_42-3469144068838265839 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:25:20.050Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:25:40.089Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:26:04.522Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:26:04.564Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.333Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.398Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.434Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.518Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.571Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.640Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T16:31:50.715Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T20:00:31.807Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-13_08_24_42-3469144068838265839.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T20:00:31.832Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T20:00:32.053Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T20:00:32.098Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-13T20:00:32.115Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-13_08_24_42-3469144068838265839 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 37m 0s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/bhnidxiqnzyb4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #432
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/432/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12683] Fix failing integration tests for Python Recommendation AI
[noreply] [BEAM-13015] Plumb the cache through contexts and transform executors.
------------------------------------------
[...truncated 366.59 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.23)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.23 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.23)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2625001 sha256=9af48a4ebcc652a287745f0dcdcc008f86d4d31a2b35da31283b68bba50e0ef6
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2de9420ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2de9422710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1212100339.1639324975.727540/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1212100339.1639324975.727540/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1212100339.1639324975.727540/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1212100339.1639324975.727540/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211212160255728522-6541'
createTime: '2021-12-12T16:02:56.875837Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-12_08_02_56-14897315009289272046'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1212100339'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-12T16:02:56.875837Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-12_08_02_56-14897315009289272046]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-12_08_02_56-14897315009289272046
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-12_08_02_56-14897315009289272046?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-12_08_02_56-14897315009289272046 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:02.885Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:08.855Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:08.924Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:08.960Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.002Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.059Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.118Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.170Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.217Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.257Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.325Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.397Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.454Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.497Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.546Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.590Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.807Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:09.923Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:10.044Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:10.070Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-12_08_02_56-14897315009289272046 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:03:15.868Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:04:00.196Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:04:24.295Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:04:24.333Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.477Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.610Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.630Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.729Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.789Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.865Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T16:10:09.952Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T20:00:34.503Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-12_08_02_56-14897315009289272046.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T20:00:34.531Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-12_08_02_56-14897315009289272046 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T20:00:35.652Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T20:00:35.712Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-12T20:00:35.744Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 58m 45s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/bwsiv5awawgti
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #431
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/431/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13424][Playground]
[aydar.zaynutdinov] [BEAM-13424][Playground]
[stranniknm] [BEAM-13426] clear log and output panels on example change
[aydar.zaynutdinov] [BEAM-13428][Playground]
[aydar.zaynutdinov] [BEAM-13424][Playground]
[daria.malkova] add type of precompiled object
[Alexey Romanenko] [BEAM-13171] Fix Duration ambiguity
[mmack] [BEAM-13410][BEAM-11788] Integration test for S3Filesystem using
[Alexey Romanenko] [BEAM-13434] Bump up Apache log4j2 vulnerability to 2.15.0
[noreply] Merge pull request #16138 from [BEAM-13343][Playground] Support go unit
[noreply] [BEAM-12560] Dataframe idxmin and idxmax implementation (#15827)
[noreply] [BEAM-12565] Dataframe compare implementation (#16027)
[noreply] [BEAM-12976] Use a map to pass all pushdown requests at once. (#16189)
------------------------------------------
[...truncated 366.67 KB...]
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.23)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.23 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.23)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2624881 sha256=9086e1f3fed0b69a3393867202cd3cf3669410bcf9339965efd2333449b4cdb8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa096045ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa096048710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1211100342.1639238252.540247/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1211100342.1639238252.540247/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1211100342.1639238252.540247/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1211100342.1639238252.540247/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211211155732541221-3838'
createTime: '2021-12-11T15:57:33.603846Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-11_07_57_32-12025270779250229772'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1211100342'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-11T15:57:33.603846Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-11_07_57_32-12025270779250229772]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-11_07_57_32-12025270779250229772
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-11_07_57_32-12025270779250229772?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-11_07_57_32-12025270779250229772 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:47.490Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.227Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.266Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.302Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.340Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.473Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.541Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.580Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.632Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.667Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.692Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.735Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.817Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.849Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.904Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:48.926Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:49.072Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:49.133Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:49.175Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:57:49.209Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-11_07_57_32-12025270779250229772 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:58:13.784Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:58:34.183Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:58:59.752Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T15:58:59.799Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:42.860Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:42.986Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:43.020Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:43.127Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:43.189Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:43.256Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T16:04:43.321Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T20:00:32.177Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-11_07_57_32-12025270779250229772.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T20:00:32.200Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T20:00:32.355Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T20:00:32.433Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-11T20:00:32.452Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-11_07_57_32-12025270779250229772 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 11s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/wvery7i2wmb4u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #430
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/430/display/redirect?page=changes>
Changes:
[slebedev] [BEAM-13384] Re-exported the metrics subpackage in
[zyichi] [BEAM-13401] Wait for worker start to reduce flakiness in
[zyichi] Fix pytest unknown markers warning.
[noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174)
[noreply] [BEAM-12561] method truncate on series and dataframe (#15833)
[noreply] [BEAM-13294] Widen key schema for all keys before use (#16158)
[noreply] [BEAM-13171] Support for stopReadTime on KafkaIO SDF (#15951)
[noreply] [BEAM-13399] Add functionality to download Beam JARs from Maven
[noreply] [BEAM-13402] Add another workaround for
------------------------------------------
[...truncated 366.20 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.23)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.28)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.23 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.23)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622870 sha256=9b79f4ae060249a3f52aa52605c24efc9b5066d1efb2975d368b2d05047638c7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1a55bb2ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1a55bb4710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1210100339.1639152543.444218/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1210100339.1639152543.444218/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1210100339.1639152543.444218/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1210100339.1639152543.444218/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211210160903445202-9888'
createTime: '2021-12-10T16:09:04.550562Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-10_08_09_03-7198455451904166359'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1210100339'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-10T16:09:04.550562Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-10_08_09_03-7198455451904166359]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-10_08_09_03-7198455451904166359
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-10_08_09_03-7198455451904166359?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-10_08_09_03-7198455451904166359 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:12.696Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.719Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.774Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.813Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.844Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.890Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:13.976Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.015Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.052Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.094Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.131Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.179Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.231Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.264Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.300Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.620Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-10_08_09_03-7198455451904166359 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.847Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:14.984Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:15.079Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:09:51.354Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:10:06.484Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:10:30.366Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:10:30.399Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.105Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.232Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.279Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.407Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.555Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.712Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T16:16:14.845Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T20:00:35.024Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-10_08_09_03-7198455451904166359.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T20:00:35.098Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T20:00:35.243Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T20:00:35.352Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-10T20:00:35.386Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-10_08_09_03-7198455451904166359 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 52m 40s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/pvdnqyohry5ey
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #429
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/429/display/redirect?page=changes>
Changes:
[stranniknm] fix playground frontend licences
[noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change
[noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines
[noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness..
[noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171)
[noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK
[noreply] [BEAM-11936] Fix errorprone warnings (#15890)
[noreply] [BEAM-13015] Start integrating a process wide cache. (#16130)
------------------------------------------
[...truncated 367.29 KB...]
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.22)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.22 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.22)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622621 sha256=025098e20cab8ddf3841dc42362b8cb75442dc3652abb9687106d4907f30ba2a
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7efd35345ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7efd35347710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1209100353.1639065423.323207/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1209100353.1639065423.323207/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1209100353.1639065423.323207/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1209100353.1639065423.323207/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211209155703324252-5379'
createTime: '2021-12-09T15:57:04.476029Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-09_07_57_03-10369477750790973907'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1209100353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-09T15:57:04.476029Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-09_07_57_03-10369477750790973907]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-09_07_57_03-10369477750790973907
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-09_07_57_03-10369477750790973907?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-09_07_57_03-10369477750790973907 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:10.585Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.036Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.066Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.118Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.157Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.193Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.271Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.304Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.337Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.371Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.405Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.442Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.474Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.505Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.545Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.720Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.777Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.827Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:13.860Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-09_07_57_03-10369477750790973907 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:24.497Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:57:58.183Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:58:22.653Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T15:58:22.683Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.531Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.625Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.658Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.723Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.837Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.902Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:04:09.974Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:48:59.428Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:49:00.041Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:49:09.483Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T16:49:23.767Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T20:00:21.902Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-09_07_57_03-10369477750790973907.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T20:00:21.931Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T20:00:22.081Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T20:00:22.246Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-09T20:00:22.272Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-09_07_57_03-10369477750790973907 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 25s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/5vh4zntndr24o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #428
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/428/display/redirect?page=changes>
Changes:
[heejong] [BEAM-13092] Adding dummy external transform translators for Dataflow
[stranniknm] [BEAM-13112]: playground embedded version
[zyichi] [BEAM-13388] Fix broken google cloud dlp test.
[relax] don't store unserializable DatasetService as a member variable
[noreply] [BEAM-13236] Properly close kinesis producer on teardown (#15955)
[noreply] Merge pull request #16150 from [BEAM-13396][Playground][Bugfix] Issues
[noreply] Merge pull request #16148 from [BEAM-13394][Playground] [Bugfix] Fix
[noreply] Avoid overriding explicit portable job submission disabling. (#16143)
[noreply] Merge pull request #16151 from [BEAM-13350][Playground] Support running
[zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of
[melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content
[msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test.
[noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up
[dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577
[noreply] [BEAM-12976] Implement pipeline visitor to get global field access in…
[noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164)
[noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support
------------------------------------------
[...truncated 366.28 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.3)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.21)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.54.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.3)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622621 sha256=99e0f8c15fe46e9cfba83576a27c74b16adcc513bf4f920ac89cfcce2f42431f
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff23c1c3ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff23c1c5710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1208100400.1638979114.129175/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1208100400.1638979114.129175/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1208100400.1638979114.129175/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1208100400.1638979114.129175/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211208155834130180-2137'
createTime: '2021-12-08T15:58:35.364914Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-08_07_58_34-15971022465025869277'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1208100400'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-08T15:58:35.364914Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-08_07_58_34-15971022465025869277]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-08_07_58_34-15971022465025869277
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-08_07_58_34-15971022465025869277?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-08_07_58_34-15971022465025869277 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:42.786Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.337Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.380Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.407Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.448Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.491Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.528Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.563Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.594Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.627Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.656Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.691Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.720Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.746Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.787Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:50.813Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:51.009Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:51.076Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:51.117Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:58:51.148Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-08_07_58_34-15971022465025869277 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:59:08.440Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:59:34.470Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:59:57.283Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T15:59:57.312Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.377Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.531Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.563Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.627Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.677Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.746Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T16:05:31.799Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T20:00:32.045Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-08_07_58_34-15971022465025869277.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T20:00:32.102Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T20:00:32.252Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T20:00:32.462Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-08T20:00:32.566Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-08_07_58_34-15971022465025869277 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1630, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 6s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/wps75m3enhyx6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #427
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/427/display/redirect?page=changes>
Changes:
[Daniel Oliveira] Minor Gradle Fixups: Removing unneeded dependency in expansion service.
[aydar.zaynutdinov] [BEAM-13363][Playground]
[aydar.zaynutdinov] [BEAM-13357][Playground]
[aydar.zaynutdinov] [BEAM-13357][Playground]
[Luke Cwik] [BEAM-13015] Simplify fake state API testing client and make it
[zyichi] [BEAM-13380] Fix broken seedjob.
[aydar.zaynutdinov] [BEAM-13357][Playground]
[noreply] Merge pull request #16137 from [BEAM-13381][Playground][Bugfix] Update
[noreply] Merge pull request #16107 from [BEAM-13332] [Playground] Add logs output
[noreply] Merge pull request #16109 from [BEAM-13330][Playground] Save Java logs
[noreply] Merge pull request #16125 from [BEAM-13322][Playground] Support Java
[melissapa] [BEAM-11758] Update basics page: Window, Watermark
------------------------------------------
[...truncated 365.92 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.21)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.1)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.2)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622596 sha256=56be960c213aab5333d5dac9b536b3b86f51779bbc37d7a6b069210ef6d5b957
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7d50a7fef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7d50a81710> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1207100355.1638892058.756267/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1207100355.1638892058.756267/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1207100355.1638892058.756267/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1207100355.1638892058.756267/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211207154738757329-2479'
createTime: '2021-12-07T15:47:39.946208Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-07_07_47_39-12985069232578445610'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1207100355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-07T15:47:39.946208Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-07_07_47_39-12985069232578445610]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-07_07_47_39-12985069232578445610
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-07_07_47_39-12985069232578445610?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-07_07_47_39-12985069232578445610 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:47.627Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.326Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.435Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.498Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.541Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.590Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.629Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.662Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.705Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.741Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.775Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.807Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.840Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.893Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.926Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:53.961Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:54.170Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:54.336Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:54.442Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:47:54.491Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-07_07_47_39-12985069232578445610 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:48:22.163Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:48:38.008Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:49:01.807Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:49:01.845Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.481Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.563Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.612Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.708Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.751Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.832Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T15:54:47.944Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-07_07_47_39-12985069232578445610 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T20:00:31.721Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-07_07_47_39-12985069232578445610.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T20:00:31.779Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T20:00:32.405Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T20:00:32.459Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-07T20:00:32.486Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 2s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/ahobybvdqibxs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #426
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/426/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16118 from [BEAM-13362][Playground] python ci cd
------------------------------------------
[...truncated 366.07 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.5)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.20)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.20 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.20)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.5.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622596 sha256=2c2fc116d922f95d599e977ed8a180c025ea8a6682cf3742b63f2a5962ae74dd
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff444a3a050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff444a3a830> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1206100348.1638807128.752357/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1206100348.1638807128.752357/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1206100348.1638807128.752357/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1206100348.1638807128.752357/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211206161208753403-4904'
createTime: '2021-12-06T16:12:09.939640Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-06_08_12_09-4205662942573614233'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1206100348'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-06T16:12:09.939640Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-06_08_12_09-4205662942573614233]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-06_08_12_09-4205662942573614233
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-06_08_12_09-4205662942573614233?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-06_08_12_09-4205662942573614233 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:31.457Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.532Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.560Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.599Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.633Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.661Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.690Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.735Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.767Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.805Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.836Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.869Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.905Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.924Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.959Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:32.980Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:33.123Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:33.189Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:33.235Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:33.265Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-06_08_12_09-4205662942573614233 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:12:43.702Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:13:16.522Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:13:44.427Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:13:44.554Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.145Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.240Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.263Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.321Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.385Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.443Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T16:19:23.522Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T20:00:30.153Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-06_08_12_09-4205662942573614233.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T20:00:30.187Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T20:00:30.332Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T20:00:30.469Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-06T20:00:30.490Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-06_08_12_09-4205662942573614233 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 49m 30s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/ai62qwxdww4jw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #425
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/425/display/redirect>
Changes:
------------------------------------------
[...truncated 366.88 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.20)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.20 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.20)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622596 sha256=dbf34cb56e015b7570a544792c1c4a46c4d52c9cedabc01e757d453107cf1037
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4ec1b48c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4ec1b49440> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1205112722.1638719170.219761/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1205112722.1638719170.219761/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1205112722.1638719170.219761/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1205112722.1638719170.219761/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211205154610220757-7236'
createTime: '2021-12-05T15:46:11.271597Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-05_07_46_10-13989375151051691016'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1205112722'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-05T15:46:11.271597Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-05_07_46_10-13989375151051691016]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-05_07_46_10-13989375151051691016
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-05_07_46_10-13989375151051691016?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-05_07_46_10-13989375151051691016 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:15.347Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.067Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.097Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.143Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.166Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.197Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.222Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.247Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.281Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.314Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.347Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.382Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.417Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.456Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.493Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.527Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.659Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.726Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.773Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:16.803Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-05_07_46_10-13989375151051691016 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:46:48.581Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:47:00.271Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:47:24.221Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:47:24.256Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.077Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.136Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.168Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.219Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.259Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.313Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T15:53:06.384Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T20:00:31.660Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-05_07_46_10-13989375151051691016.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T20:00:31.691Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T20:00:31.855Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T20:00:31.982Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-05T20:00:32.002Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-05_07_46_10-13989375151051691016 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 15m 31s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/4pcnrvhenfynw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #424
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/424/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Revert "Revert "Allow wildcards for java class lookup transform
[Robert Bradshaw] Move stand-alone expansion service jar into its own project.
[noreply] Merge pull request #16117 from [BEAM-13368][Playground][Bugfix] Fix CI
[noreply] Run python SpannerIO IT with python 3.7 only to avoid overload spanner
[noreply] Merge pull request #15378 from [RFC] Define and document per-key
[noreply] Bump python containers to beam-master-20211202 (#16129)
[noreply] [BEAM-13354, BEAM-13015, BEAM-12802, BEAM-12588] Support prefetch for
[zyichi] Fix failing RecommendationAICatalogItemIT
[Valentyn Tymofieiev] Clarify instructions on how to get contributor list.
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in IO (#16036)
------------------------------------------
[...truncated 365.39 KB...]
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.0)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.20)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.20 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.20)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.9)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2622596 sha256=b678aede1610be681141129b30f2a0dc298f2e486ec7e41043eb9547bbc5347e
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211202" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f643df72c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f643df73440> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1204100354.1638635112.556332/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1204100354.1638635112.556332/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1204100354.1638635112.556332/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1204100354.1638635112.556332/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211204162512557333-3263'
createTime: '2021-12-04T16:25:14.333159Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-04_08_25_12-10215149657968622423'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1204100354'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-04T16:25:14.333159Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-04_08_25_12-10215149657968622423]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-04_08_25_12-10215149657968622423
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-04_08_25_12-10215149657968622423?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-04_08_25_12-10215149657968622423 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:19.352Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.524Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.567Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.599Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.630Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.673Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.707Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.741Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.778Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.810Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.844Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.896Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.939Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:21.973Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.005Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.040Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.223Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.311Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.358Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:22.387Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-04_08_25_12-10215149657968622423 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:25:29.938Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:26:10.913Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:26:34.787Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:26:34.821Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.284Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.366Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.418Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.511Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.603Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.659Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T16:32:16.777Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-04_08_25_12-10215149657968622423 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T20:00:29.750Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-04_08_25_12-10215149657968622423.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T20:00:29.782Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T20:00:29.912Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T20:00:30.022Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-04T20:00:30.053Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 36m 26s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/6ifiuwptc4znc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #423
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/423/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13329][Playground]
[aydar.zaynutdinov] [BEAM-13329][Playground]
[alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings
[ilya.kozyrev] Fix pylint issues and apply yapf with Beam config
[ilya.kozyrev] fix white spaces
[noreply] Don't pin a particular version of Tensorflow. (#16102)
[noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI
[noreply] [BEAM-13288] improve logging for no rows present error (#16096)
------------------------------------------
[...truncated 366.20 KB...]
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.21.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.19)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.31.0)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.19 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.19)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2620759 sha256=702789d7708065349269342c70d991474c7030a8b12903de6bf9ce4922a9178b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3a8301b8c0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3a8301c0e0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1203100350.1638548402.635779/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1203100350.1638548402.635779/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1203100350.1638548402.635779/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1203100350.1638548402.635779/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211203162002636809-4471'
createTime: '2021-12-03T16:20:03.766496Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-03_08_20_03-16437846717274973345'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1203100350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-03T16:20:03.766496Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-03_08_20_03-16437846717274973345]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-03_08_20_03-16437846717274973345
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-03_08_20_03-16437846717274973345?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-03_08_20_03-16437846717274973345 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:08.537Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.312Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.341Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.371Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.394Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.433Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.461Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.484Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.515Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.547Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.581Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.603Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.638Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.675Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.713Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.742Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.908Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:09.970Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:10.017Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:10.048Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-03_08_20_03-16437846717274973345 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:31.827Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:20:54.734Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:21:18.550Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:21:18.582Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.128Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.217Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.239Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.296Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.364Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.429Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T16:27:00.492Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T20:00:31.760Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-03_08_20_03-16437846717274973345.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T20:00:31.782Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T20:00:31.918Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T20:00:32.012Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-03T20:00:32.037Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-03_08_20_03-16437846717274973345 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 41m 39s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/ldf6wi3qnpkss
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #422
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/422/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Require explicit designation of proxy partitioning specification.
[Robert Bradshaw] Elementwise transforms preserve all partitionings.
[aydar.zaynutdinov] [BEAM-13360][Playground]
[noreply] Merge pull request #16011 from [BEAM-12164] Add models for Spanner
[noreply] [BEAM-13335] Use signed int64 range for DataFrame read unique indexes
[iyi] Add test case for untriggered expansion with temp tables.
[zyichi] Fix small typo in ProcessBundleHandlerTest.java
[noreply] Merge pull request #16079 from [BEAM-13241] [Playground] Frontend
[noreply] Ricardo Case Study (#16087)
[noreply] Merge pull request #16086 from [BEAM-13258][Playground] Get and store
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Runners (#16035)
[noreply] [BEAM-8123] Add cloudpickle as optional library (#15472)
[noreply] [BEAM-12572] Beam python examples continuously exercised on at least 2
------------------------------------------
[...truncated 367.17 KB...]
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: cloudpickle<3,>=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.18)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.18 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.18)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2620759 sha256=706bb8a7be43e69d7139b3b60c02a6fbc160d2735f96aff2aceb158ddca95d0d
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7b57d0a5f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7b57d0add0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1202100340.1638460621.160271/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1202100340.1638460621.160271/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1202100340.1638460621.160271/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1202100340.1638460621.160271/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211202155701161239-5797'
createTime: '2021-12-02T15:57:02.696778Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-02_07_57_01-13717783728299845673'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1202100340'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-02T15:57:02.696778Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-02_07_57_01-13717783728299845673]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-02_07_57_01-13717783728299845673
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-02_07_57_01-13717783728299845673?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-02_07_57_01-13717783728299845673 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:10.535Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.446Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.479Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.504Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.537Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.590Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.619Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.642Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.671Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.704Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.728Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.760Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.805Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.840Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.864Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:11.890Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:12.041Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:12.109Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:12.146Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:12.180Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-02_07_57_01-13717783728299845673 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:57:33.496Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:58:00.319Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:58:23.110Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T15:58:23.143Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.667Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.761Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.786Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.864Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.916Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:09.985Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T16:04:10.058Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T20:00:31.464Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-02_07_57_01-13717783728299845673.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T20:00:31.498Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T20:00:31.641Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T20:00:31.713Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-02T20:00:31.739Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-02_07_57_01-13717783728299845673 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 41s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/js7tuld5sfd3g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #421
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/421/display/redirect?page=changes>
Changes:
[mmack] [BEAM-11494][BEAM-11821] FileIO stops overwriting files on retries (AWS
[Pablo Estrada] [py] Supporting ignore_unknown_values for WriteToBigQuery
[noreply] Merge pull request #16072 from [BEAM-13135][Playground] Add function to
[noreply] Merge pull request #16084 from [BEAM-13347][Playground] [Bugfix] Backend
[noreply] Merge pull request #15987 from [BEAM-13242] Allow values with smaller
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions (#16033)
[noreply] Merge pull request #15936 from [BEAM-13351] FhirIO GetPatientEverything
[avilovpavel6] Add fmt preparator for go sdk
[noreply] Merge pull request #16073 from [BEAM-13267][Playground] Implement
[ningkang0957] Updated screen diff test for Interactive Beam
[noreply] [BEAM-13282][Playground] Create go dockerfile #16049
[noreply] Merge pull request #16085 from [Beam-13336][Playground] Refactor
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions-sql (#16034)
[noreply] [BEAM-13193] Support
[noreply] Merge pull request #16090: [BEAM-13352] Add transient qualifier to
------------------------------------------
[...truncated 364.69 KB...]
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.17)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.24.0,>=1.23.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.17)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2615195 sha256=239c0576bc924326831cb652295d5198a84a126811eb1a460899ef9119e23ba7
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe316ae47a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe316ae4f80> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1201100350.1638373894.336649/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1201100350.1638373894.336649/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1201100350.1638373894.336649/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1201100350.1638373894.336649/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211201155134337618-8530'
createTime: '2021-12-01T15:51:35.538364Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-12-01_07_51_34-1942996983061382481'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1201100350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-12-01T15:51:35.538364Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-12-01_07_51_34-1942996983061382481]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-12-01_07_51_34-1942996983061382481
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-12-01_07_51_34-1942996983061382481?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-01_07_51_34-1942996983061382481 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:40.236Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:40.936Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.040Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.102Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.138Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.190Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.235Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.276Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.309Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.346Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.390Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.421Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.450Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.480Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.517Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.552Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.768Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.868Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.925Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:41.965Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-01_07_51_34-1942996983061382481 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:51:53.055Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:52:29.017Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:52:52.683Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:52:52.723Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.410Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.481Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.515Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.597Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.655Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.711Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T15:58:39.779Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T20:00:32.214Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-12-01_07_51_34-1942996983061382481.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T20:00:32.247Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T20:00:32.434Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T20:00:32.565Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-12-01T20:00:32.585Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-12-01_07_51_34-1942996983061382481 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 10m 7s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/xe2avjgdyufow
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #420
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/420/display/redirect?page=changes>
Changes:
[kileysok] Increase timeout for Dataflow Streaming VR
[Kyle Weaver] [BEAM-13220] Update release instructions.
[noreply] Merge pull request #16050 from [BEAM-13307][Playground] Support Python
[noreply] Merge pull request #16071 from [BEAM-13306][Playground] Add using of
[noreply] [BEAM-11943] Fix wrong Klio logo being displayed (#16078)
[noreply] Merge pull request #15954 from [BEAM-960][BEAM-1675] Improvements to
[noreply] [BEAM-13284] Respect expiration time for all Redis write methods.
[noreply] [BEAM-12697] Add SBE module and initial classes (#15733)
[noreply] [Playground][Beam-13316][Bugfix] Fix Playground Frontend Precommit
[Kyle Weaver] [BEAM-13337] Periodically delete stale spanner databases.
[noreply] [BEAM-13115][Playground] Security – Mock Storage (#16047)
[noreply] [BEAM-13335] Use shorter, numerical indices for dataframe reads.
[noreply] Be more conservative about rebatching. (#16058)
[noreply] [BEAM-12587] Allow None in Python's Any logical type. (#16055)
[melissapa] [BEAM-11758] Update basics page: Trigger, State and timers
------------------------------------------
[...truncated 363.76 KB...]
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.2)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.16)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.16 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.16)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614837 sha256=74bb3c471f4f8b3a9936eaac7e62c501f02e26923f82b386a163020de43bb8e8
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f71783715f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7178371dd0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1130100350.1638287921.828859/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1130100350.1638287921.828859/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1130100350.1638287921.828859/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1130100350.1638287921.828859/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211130155841829824-7662'
createTime: '2021-11-30T15:58:42.903757Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-30_07_58_42-11739530932851500000'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1130100350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-30T15:58:42.903757Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-30_07_58_42-11739530932851500000]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-30_07_58_42-11739530932851500000
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-30_07_58_42-11739530932851500000?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-30_07_58_42-11739530932851500000 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:47.827Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.408Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.441Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.463Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.493Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.535Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.561Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.585Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.608Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.642Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.662Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.689Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.720Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.746Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.785Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.819Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:48.963Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:49.047Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:49.103Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:58:49.139Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-30_07_58_42-11739530932851500000 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:59:02.561Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:59:33.931Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:59:58.444Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T15:59:58.479Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.679Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.751Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.772Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.837Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.922Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:52.983Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T16:05:53.041Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T20:00:31.857Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-30_07_58_42-11739530932851500000.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T20:00:31.891Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T20:00:32.038Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T20:00:32.116Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-30T20:00:32.153Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-30_07_58_42-11739530932851500000 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 3m 0s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/vogowcfy6p3i2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #419
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/419/display/redirect>
Changes:
------------------------------------------
[...truncated 364.36 KB...]
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.15)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.15 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.15)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.4.0)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=b933da7b6b9eaec60d05849d7cc08f8006285f9bc7d2bb5627e117db6b7cf426
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc86eb644d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc86eb64cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1129100334.1638202889.656572/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1129100334.1638202889.656572/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1129100334.1638202889.656572/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1129100334.1638202889.656572/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211129162129657526-4376'
createTime: '2021-11-29T16:21:30.801317Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-29_08_21_30-10991186721732336368'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1129100334'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-29T16:21:30.801317Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-29_08_21_30-10991186721732336368]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-29_08_21_30-10991186721732336368
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-29_08_21_30-10991186721732336368?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-29_08_21_30-10991186721732336368 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:34.840Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.395Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.434Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.473Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.508Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.537Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.569Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.598Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.625Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.660Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.688Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.718Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.755Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.788Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.817Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.849Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-29_08_21_30-10991186721732336368 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:35.987Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:36.077Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:36.126Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:21:36.155Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:22:04.639Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:22:20.768Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:22:45.248Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:22:45.280Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.250Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.311Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.344Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.399Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.448Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.514Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T16:28:38.563Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T20:00:43.968Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-29_08_21_30-10991186721732336368.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T20:00:44.005Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T20:00:44.124Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T20:00:44.482Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-29T20:00:44.523Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-29_08_21_30-10991186721732336368 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 40m 29s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/jttfdgna2azgm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #418
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/418/display/redirect?page=changes>
Changes:
[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting
[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting
[noreply] Merge pull request #16074 from [BEAM-13323][Playground] [Bugfix] Fix CI
------------------------------------------
[...truncated 366.39 KB...]
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.14)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.14)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=ee7e2d93dcdae221766f20f3a78f1305d25c2e4f494a4dc6c7e7dcfeab1e6e71
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8dba39e4d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8dba39ecb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1128100353.1638115014.922189/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1128100353.1638115014.922189/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1128100353.1638115014.922189/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1128100353.1638115014.922189/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211128155654923177-6219'
createTime: '2021-11-28T15:56:56.001578Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-28_07_56_55-16511090458671233930'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1128100353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-28T15:56:56.001578Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-28_07_56_55-16511090458671233930]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-28_07_56_55-16511090458671233930
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-28_07_56_55-16511090458671233930?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-28_07_56_55-16511090458671233930 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:00.517Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.473Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.511Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.541Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.571Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.618Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.656Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.678Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.709Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.750Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.799Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.832Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.863Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.928Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.960Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:01.988Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:02.125Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:02.184Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:02.220Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:02.258Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-28_07_56_55-16511090458671233930 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:11.674Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:57:52.443Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:58:16.853Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T15:58:16.875Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:02.756Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:02.842Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:02.888Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:02.953Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:03.011Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:03.073Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T16:04:03.139Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T20:00:35.793Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-28_07_56_55-16511090458671233930.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T20:00:35.822Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T20:00:35.960Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T20:00:36.136Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-28T20:00:36.159Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-28_07_56_55-16511090458671233930 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 4m 51s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/yfxi57zbujvmw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #417
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/417/display/redirect>
Changes:
------------------------------------------
[...truncated 365.62 KB...]
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.14)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.14)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.3.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=df4065f3a841f3c25a81fb90ba476a240fe4b2841e5e9bf4e72cbf765e78fab2
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f841b5334d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f841b533cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1127100347.1638028416.806801/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1127100347.1638028416.806801/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1127100347.1638028416.806801/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1127100347.1638028416.806801/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211127155336807842-5258'
createTime: '2021-11-27T15:53:38.912388Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-27_07_53_37-15712124873148415564'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1127100347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-27T15:53:38.912388Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-27_07_53_37-15712124873148415564]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-27_07_53_37-15712124873148415564
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-27_07_53_37-15712124873148415564?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-27_07_53_37-15712124873148415564 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.014Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.654Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.684Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.710Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.735Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.765Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.807Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.829Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.869Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.894Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.916Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:43.977Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.006Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.031Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.059Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.191Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.260Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.309Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:53:44.337Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-27_07_53_37-15712124873148415564 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:14.222Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:28.563Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:28.591Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:38.842Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:57.291Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T15:54:57.325Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.681Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.762Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.785Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.838Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.889Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:24.942Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T16:00:25.019Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T20:00:34.740Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-27_07_53_37-15712124873148415564.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T20:00:34.764Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T20:00:34.900Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T20:00:34.959Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-27T20:00:34.980Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-27_07_53_37-15712124873148415564 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 8m 7s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/2vpvcgzia5aem
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #416
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/416/display/redirect>
Changes:
------------------------------------------
[...truncated 364.33 KB...]
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.13)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.13 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.13)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=a83778691ff9b6cfa8d9c9f7f04cfff574a2056110724cbdb29064a7379c72da
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f381e6804d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f381e680cb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1126100354.1637941642.668876/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1126100354.1637941642.668876/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1126100354.1637941642.668876/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1126100354.1637941642.668876/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211126154722669988-6489'
createTime: '2021-11-26T15:47:23.693667Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-26_07_47_23-993343210265935941'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1126100354'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-26T15:47:23.693667Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-26_07_47_23-993343210265935941]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-26_07_47_23-993343210265935941
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-26_07_47_23-993343210265935941?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-26_07_47_23-993343210265935941 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:29.407Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.375Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.452Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.485Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.520Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.563Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.597Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.641Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.690Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.713Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.746Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.776Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.808Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.837Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.873Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:30.898Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:31.049Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:31.121Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:31.167Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:47:31.197Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-26_07_47_23-993343210265935941 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:48:06.195Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:48:17.931Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:48:40.099Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:48:40.127Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.263Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.322Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.353Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.421Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.466Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.547Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T15:54:23.618Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T20:00:36.643Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-26_07_47_23-993343210265935941.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T20:00:36.667Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T20:00:36.814Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T20:00:37.839Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-26T20:00:37.869Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-26_07_47_23-993343210265935941 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 22s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/rnpd6hpm4yesc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #415
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/415/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify
[Luke Cwik] [BEAM-13015] Simplify DataStreamsDecoder by moving hasNext logic to next
------------------------------------------
[...truncated 363.85 KB...]
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.13)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.13 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.13)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.8)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=6424433559dc76594820f6333eee27a4d7f3c4cec403a7d81b9a61a723238b0b
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f356a3ef4d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f356a3efcb0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1125100353.1637855774.336753/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1125100353.1637855774.336753/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1125100353.1637855774.336753/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1125100353.1637855774.336753/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211125155614337764-1674'
createTime: '2021-11-25T15:56:15.461395Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-25_07_56_14-17193576963385593131'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1125100353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-25T15:56:15.461395Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-25_07_56_14-17193576963385593131]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-25_07_56_14-17193576963385593131
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-25_07_56_14-17193576963385593131?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-25_07_56_14-17193576963385593131 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:19.552Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.292Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.320Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.340Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.363Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.390Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.404Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.428Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.448Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.471Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.492Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.515Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.538Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.562Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.585Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.612Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.744Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.792Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.829Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:20.851Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-25_07_56_14-17193576963385593131 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:56:30.585Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:57:05.500Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:57:28.151Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T15:57:28.191Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.490Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.572Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.593Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.648Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.705Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.771Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T16:03:08.850Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T20:00:32.174Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-25_07_56_14-17193576963385593131.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T20:00:32.200Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T20:00:32.382Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T20:00:32.544Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-25T20:00:32.564Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-25_07_56_14-17193576963385593131 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 5m 26s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/jh6qjesrribgq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_SideInput_Dataflow_Batch #414
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/414/display/redirect?page=changes>
Changes:
[noreply] Increase the timeout for LoadTest Jenkins jobs 4h->12h.
[noreply] [BEAM-13073] UseParallelGC on Java container
[noreply] Merge pull request #16002 from [BEAM-13101] [Playground] load examples
[noreply] Merge pull request #16016 from [BEAM-13089][Playground] Enable
[noreply] Merge pull request #16046 from [BEAM-13295] [Playground] [Bugfix]
[noreply] Merge pull request #16048 from [Beam-13256] [Playground] Implement an
[noreply] Merge pull request #16026 from [BEAM-13208][Playground] Separate stderr
[noreply] Append to state in chunks. (#15983)
[Luke Cwik] [BEAM-13313] Fix WindowingStrategy proto to use snake case field names.
[noreply] [BEAM-12829] Never copy .gogradle/** for Python. (#16059)
------------------------------------------
[...truncated 364.29 KB...]
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.6.4)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.7)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.42.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.12.1)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.19.1)
Requirement already satisfied: proto-plus<2,>=1.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.8)
Requirement already satisfied: pyarrow<7.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0.1)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.10.0.2)
Requirement already satisfied: pandas<1.4,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.4)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.12)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.0.0)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<7.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (6.0)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.4.27)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.9.2)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.4.2)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.20.1)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.30.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (2.10.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<4,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (3.3.1)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.36.0.dev0) (0.2.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.36.0.dev0) (1.16.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.21)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (36.0.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (1.23.12)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.36.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.36.0.dev0) (0.16.3)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (59.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.36.0.dev0) (4.8)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (21.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.31.4)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.3.23)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.36.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.36.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.36.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.36.0.dev0) (5.8.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.36.0.dev0) (0.4.8)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.11.0)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (1.4.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.2.5)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (8.12.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (0.13.1)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (2.1.2)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (21.2.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.9.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (2.0.7)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.36.0.dev0) (3.3)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.36.0.dev0) (1.1.2)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (5.0.3)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.13.3)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (2.1.0)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.36.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.7.1)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (0.6.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (1.3.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.36.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (2.21)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.36.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.36.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.36.0.dev0-py3-none-any.whl size=2614323 sha256=58412f5cc9b002da076228fd2e85dd98e832b31ed3e019327e0275525c093f50
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.36.0.dev0
Uninstalling apache-beam-2.36.0.dev0:
Successfully uninstalled apache-beam-2.36.0.dev0
Successfully installed apache-beam-2.36.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:100 out of 10000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211116" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4fd75f30e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4fd75f38c0> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1124100403.1637768887.345198/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1124100403.1637768887.345198/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1124100403.1637768887.345198/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-9-1124100403.1637768887.345198/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=iter', '--access_percentage=1', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211124154807346155-2865'
createTime: '2021-11-24T15:48:08.384597Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-11-24_07_48_07-11606931131778664280'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-9-1124100403'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-11-24T15:48:08.384597Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-11-24_07_48_07-11606931131778664280]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-11-24_07_48_07-11606931131778664280
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-24_07_48_07-11606931131778664280?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-24_07_48_07-11606931131778664280 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:12.715Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.584Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.612Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.700Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.722Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.764Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.802Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.835Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.859Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.889Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.916Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.950Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:13.975Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.003Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.032Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.053Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.226Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.504Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.564Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:14.585Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-24_07_48_07-11606931131778664280 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:48:46.028Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:49:05.878Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:49:34.008Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:49:34.035Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:10.644Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:10.775Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:10.807Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:10.902Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:10.975Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:11.045Z: JOB_MESSAGE_DEBUG: Value "ParDo(SequenceSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T15:55:11.098Z: JOB_MESSAGE_BASIC: Executing operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T20:00:34.488Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2021-11-24_07_48_07-11606931131778664280.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T20:00:34.515Z: JOB_MESSAGE_BASIC: Finished operation ParDo(SequenceSideInputTestDoFn)/ParDo(SequenceSideInputTestDoFn)+Collect end time metrics
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T20:00:34.659Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T20:00:34.734Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-11-24T20:00:34.812Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-11-24_07_48_07-11606931131778664280 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/sideinput_test.py",> line 216, in <module>
SideInputTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1628, in wait_until_finish
'Job did not reach to a terminal state after waiting indefinitely.')
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 14m 2s
5 actionable tasks: 2 executed, 3 up-to-date
Publishing build scan...
https://gradle.com/s/r7kpfjf2xr3dc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 413 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 413 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/413/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 412 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 412 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/412/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 411 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 411 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/411/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 410 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 410 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/410/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 409 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 409 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/409/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 408 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 408 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/408/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 407 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 407 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/407/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 406 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 406 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/406/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 405 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 405 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/405/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 404 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 404 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/404/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 403 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 403 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/403/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 402 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 402 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/402/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 401 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 401 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/401/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 400 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 400 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/400/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 399 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 399 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/399/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 398 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 398 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/398/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 397 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 397 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/397/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 396 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 396 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/396/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 395 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 395 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/395/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 394 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 394 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/394/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 393 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 393 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/393/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 392 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 392 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/392/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 391 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 391 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/391/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 390 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 390 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/390/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 389 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 389 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/389/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 388 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 388 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/388/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 387 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 387 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/387/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 386 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 386 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/386/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 385 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 385 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/385/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 384 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 384 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/384/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 383 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 383 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/383/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 382 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 382 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/382/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 381 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 381 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/381/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 380 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 380 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/380/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 379 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 379 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/379/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 378 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 378 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/378/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 377 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 377 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/377/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 376 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 376 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/376/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 375 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 375 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/375/ to view the results.
Build failed in Jenkins:
beam_LoadTests_Python_SideInput_Dataflow_Batch #374
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/374/display/redirect?page=changes>
Changes:
[ilya.kozyrev] Add java executor to compile java code in created file system
[zyichi] Use daemon thread to report lull operation and move code to
[noreply] Merge pull request #15722 from [BEAM-13022] [Playground] Run code on the
[Luke Cwik] [BEAM-12640] Skip checkerframework on generated code/test libraries for
[Luke Cwik] [BEAM-13015] Add tiny bundle processing benchmark for Java SDK harness.
[noreply] [BEAM-12922] Test MapState putIfAbsent with no read (#15704)
[Robert Bradshaw] [BEAM-13053] Avoid runner v2 when streaming engine explicitly disabled.
[noreply] [Go SDK] Small typo fixes. (#15725)
[noreply] [BEAM-13038] Migrate to use a context object within
[noreply] [BEAM-12135] Avoid windowing overhead in Spark global GBK. (#15637)
[noreply] [BEAM-12907] Run DataFrame API tests with multiple pandas versions
[vachan] Remove unnecessary ERROR logging.
------------------------------------------
[...truncated 328.85 KB...]
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: avro-python3!=1.9.2,<1.10.0,>=1.8.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.9.2.1)
Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.7)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<2,>=0.21.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.4.5)
Requirement already satisfied: future<1.0.0,>=0.18.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.18.2)
Requirement already satisfied: grpcio<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.41.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.6.0)
Requirement already satisfied: httplib2<0.20.0,>=0.8 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.19.1)
Requirement already satisfied: numpy<1.21.0,>=1.14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.20.3)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (3.12.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (4.1.3)
Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (3.17.3)
Requirement already satisfied: pyarrow<6.0.0,>=0.15.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (5.0.0)
Requirement already satisfied: pydot<2,>=1.2.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.4.2)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.8.2)
Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2021.3)
Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.26.0)
Requirement already satisfied: typing-extensions<4,>=3.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (3.10.0.2)
Requirement already satisfied: orjson<4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (3.6.4)
Requirement already satisfied: cachetools<5,>=3.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (4.2.4)
Requirement already satisfied: google-apitools<0.5.32,>=0.5.31 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.5.31)
Requirement already satisfied: google-auth<3,>=1.18.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.35.0)
Requirement already satisfied: google-cloud-datastore<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.15.3)
Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-bigquery<3,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.28.1)
Requirement already satisfied: google-cloud-bigquery-storage>=2.6.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.9.1)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.7.2)
Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.7.0)
Requirement already satisfied: google-cloud-spanner<2,>=1.13.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.19.1)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<2,>=0.12.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.16.1)
Requirement already satisfied: google-cloud-vision<2,>=0.38.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-recommendations-ai<=0.2.0,>=0.1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.2.0)
Requirement already satisfied: freezegun>=0.3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.1.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.0.0)
Requirement already satisfied: pandas<2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.3.3)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (0.7.5)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<6.0.0,>=3.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (5.4.1)
Requirement already satisfied: requests_mock<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.9.3)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (4.6.11)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.34.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.4.2)
Requirement already satisfied: sqlalchemy<2.0,>=1.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.4.25)
Requirement already satisfied: psycopg2-binary<3.0.0,>=2.8.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (2.9.1)
Requirement already satisfied: testcontainers<4.0.0,>=3.0.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (3.4.2)
Requirement already satisfied: boto3>=1.9 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.18.62)
Requirement already satisfied: azure-storage-blob>=12.3.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (12.9.0)
Requirement already satisfied: azure-core>=1.7.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from apache-beam==2.35.0.dev0) (1.19.0)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.35.0.dev0) (1.16.0)
Requirement already satisfied: cryptography>=2.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (35.0.0)
Requirement already satisfied: msrest>=0.6.21 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (0.6.21)
Requirement already satisfied: botocore<1.22.0,>=1.21.62 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.35.0.dev0) (1.21.62)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.35.0.dev0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from boto3>=1.9->apache-beam==2.35.0.dev0) (0.10.0)
Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.35.0.dev0) (0.16.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.35.0.dev0) (0.2.8)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.35.0.dev0) (58.2.0)
Requirement already satisfied: rsa<5,>=3.1.4 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.35.0.dev0) (4.7.2)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (21.0)
Requirement already satisfied: google-api-core[grpc]<3.0.0dev,>=1.29.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (1.31.3)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (2.0.3)
Requirement already satisfied: proto-plus>=1.10.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (1.19.5)
Requirement already satisfied: libcst>=0.2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery-storage>=2.6.3->apache-beam==2.35.0.dev0) (0.3.21)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigtable<2,>=0.31.1->apache-beam==2.35.0.dev0) (0.12.3)
Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam==2.35.0.dev0) (0.6.2)
Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.35.0.dev0) (2.4.7)
Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from mock<3.0.0,>=1.0.1->apache-beam==2.35.0.dev0) (5.6.0)
Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from oauth2client<5,>=2.0.1->apache-beam==2.35.0.dev0) (0.4.8)
Requirement already satisfied: more-itertools>=4.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (8.10.0)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (1.10.0)
Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (21.2.0)
Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (0.2.5)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (2.1.1)
Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (0.13.1)
Requirement already satisfied: atomicwrites>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (1.4.0)
Requirement already satisfied: pytest-forked in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.35.0.dev0) (1.3.0)
Requirement already satisfied: execnet>=1.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest-xdist<2,>=1.29.0->apache-beam==2.35.0.dev0) (1.9.0)
Requirement already satisfied: idna<4,>=2.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.35.0.dev0) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.35.0.dev0) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.35.0.dev0) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam==2.35.0.dev0) (1.26.7)
Requirement already satisfied: greenlet!=0.4.17 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from sqlalchemy<2.0,>=1.3->apache-beam==2.35.0.dev0) (1.1.2)
Requirement already satisfied: wrapt in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.35.0.dev0) (1.13.2)
Requirement already satisfied: deprecation in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.35.0.dev0) (2.1.0)
Requirement already satisfied: docker in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from testcontainers<4.0.0,>=3.0.3->apache-beam==2.35.0.dev0) (5.0.3)
Requirement already satisfied: cffi>=1.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (1.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc]<3.0.0dev,>=1.29.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (1.53.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3,>=1.6.0->apache-beam==2.35.0.dev0) (1.3.0)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.35.0.dev0) (3.6.0)
Requirement already satisfied: typing-inspect>=0.4.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.35.0.dev0) (0.7.1)
Requirement already satisfied: requests-oauthlib>=0.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (1.3.0)
Requirement already satisfied: isodate>=0.6.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (0.6.0)
Requirement already satisfied: websocket-client>=0.32.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from docker->testcontainers<4.0.0,>=3.0.3->apache-beam==2.35.0.dev0) (1.2.1)
Requirement already satisfied: pycparser in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from cffi>=1.12->cryptography>=2.1.4->azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (2.20)
Requirement already satisfied: oauthlib>=3.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob>=12.3.2->apache-beam==2.35.0.dev0) (3.1.1)
Requirement already satisfied: mypy-extensions>=0.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from typing-inspect>=0.4.0->libcst>=0.2.5->google-cloud-bigquery-storage>=2.6.3->apache-beam==2.35.0.dev0) (0.4.3)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.35.0.dev0-py3-none-any.whl size=2575941 sha256=97613219784e4929e4dc336767a392035f496670f8fca8cab6726793977d2233
Stored in directory: /home/jenkins/.cache/pip/wheels/c1/f5/a3/8f5ec48b796c1b690bd340ec7688fb88886afa83d7f14ff737
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.35.0.dev0
Uninstalling apache-beam-2.35.0.dev0:
Successfully uninstalled apache-beam-2.35.0.dev0
Successfully installed apache-beam-2.35.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:root:990 out of 1000 total elements in each window will be accessed.
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211011
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37:beam-master-20211011" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1676e62e18> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1676e63620> ====================
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/loadtests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-1015100402.1634307101.330794/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-1015100402.1634307101.330794/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-1015100402.1634307101.330794/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/loadtests/load-tests-python-dataflow-batch-sideinput-8-1015100402.1634307101.330794/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['shuffle_mode=appliance', '--side_input_type=dict', '--access_percentage=99', '--window_count=1000']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20211015141141331795-3080'
createTime: '2021-10-15T14:11:42.360429Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-10-15_07_11_41-2338310864242498199'
location: 'us-central1'
name: 'load-tests-python-dataflow-batch-sideinput-8-1015100402'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-10-15T14:11:42.360429Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2021-10-15_07_11_41-2338310864242498199]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2021-10-15_07_11_41-2338310864242498199
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-15_07_11_41-2338310864242498199?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-10-15_07_11_41-2338310864242498199 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:46.310Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.021Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.055Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.073Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.110Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.148Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.176Z: JOB_MESSAGE_DETAILED: Fusing consumer Get synthetic SDF options into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.210Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect start time metrics into Apply windows
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.243Z: JOB_MESSAGE_DETAILED: Fusing consumer Assign timestamps into Create/Read
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.282Z: JOB_MESSAGE_DETAILED: Fusing consumer Apply windows into Assign timestamps
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.304Z: JOB_MESSAGE_DETAILED: Fusing consumer Generate input into Get synthetic SDF options
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.327Z: JOB_MESSAGE_DETAILED: Fusing consumer Collect end time metrics into ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.351Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.372Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.397Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.431Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.564Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.631Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.669Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.697Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-10-15_07_11_41-2338310864242498199 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:08.619Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:36.625Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:36.648Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:46.974Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:59.380Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:59.410Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.470Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.522Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.553Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.605Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.652Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.709Z: JOB_MESSAGE_DEBUG: Value "ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.773Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
Terminated
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=dd409103-55ba-478f-8ac9-49eafb0c4fc1, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 26671
log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-26671.out.log
----- Last 20 lines from daemon log file - daemon-26671.out.log -----
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.564Z: JOB_MESSAGE_DEBUG: Executing wait step start4
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.631Z: JOB_MESSAGE_BASIC: Executing operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.669Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:11:47.697Z: JOB_MESSAGE_BASIC: Starting 10 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2021-10-15_07_11_41-2338310864242498199 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:08.619Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:36.625Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 9 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:36.648Z: JOB_MESSAGE_DETAILED: Resized **** pool to 9, though goal was 10. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:46.974Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 10 based on the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:59.380Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:12:59.410Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.470Z: JOB_MESSAGE_BASIC: Finished operation Create/Read+Assign timestamps+Apply windows+Get synthetic SDF options+Collect start time metrics+Generate input
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.522Z: JOB_MESSAGE_DEBUG: Value "Collect start time metrics.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.553Z: JOB_MESSAGE_DEBUG: Value "Generate input.out" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.605Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.652Z: JOB_MESSAGE_BASIC: Finished operation ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0)
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.709Z: JOB_MESSAGE_DEBUG: Value "ParDo(MappingSideInputTestDoFn)/_UnpickledSideInput(Generate input.out.0).output" materialized.
INFO:apache_beam.runners.dataflow.dataflow_runner:2021-10-15T14:17:26.773Z: JOB_MESSAGE_BASIC: Executing operation ParDo(MappingSideInputTestDoFn)/ParDo(MappingSideInputTestDoFn)+Collect end time metrics
Terminated
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 373 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 373 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/373/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 372 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 372 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/372/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 371 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 371 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/371/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 370 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 370 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/370/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 369 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 369 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/369/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 368 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 368 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/368/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 367 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 367 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/367/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 366 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 366 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/366/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 365 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 365 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/365/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 364 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 364 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/364/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 363 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 363 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/363/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 362 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 362 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/362/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 361 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 361 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/361/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 360 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 360 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/360/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 359 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 359 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/359/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 358 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 358 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/358/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 357 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 357 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/357/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 356 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 356 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/356/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 355 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 355 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/355/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 354 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 354 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/354/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 353 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 353 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/353/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 352 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 352 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/352/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 351 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 351 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/351/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 350 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 350 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/350/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 349 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 349 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/349/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 348 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 348 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/348/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 347 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 347 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/347/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 346 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 346 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/346/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 345 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 345 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/345/ to view the results.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 344 -
Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_SideInput_Dataflow_Batch - Build # 344 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_SideInput_Dataflow_Batch/344/ to view the results.